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IN RE PATENT APPLICATION OF ) 
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Craig David Weissman, Gregory Vincent ) 

Walsh, Eliot Leonard Wegbreit ) 
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) Group Art Unit: Not yet assigned 
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Filing Date: Filed Herewith ) 
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Title: Method and Apparatus for ) 

Creating a Well-Formed Database ) 

System Using a Computer ) 
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POWER OF ATTORNEY BY ASSIGNEE 
TO EXCLUSION OF INVENTOR UNDER 37 C.F.R. § 3.71 
WITH REVOCATION OF PRIOR POWERS 

Commissioner of Patents 
and Trademarks 
Washington, D.C. 20231 

Sir: 

The undersigned ASSIGNEE of the entire interest in the above-identified application for 
letters patent hereby appoints Paul Davis, Reg. No. 29,294; Mark A. Haynes, Reg. No. 
30,846; David J. Weitz, Reg. No. 38,362; Kent R. Richardson, Reg. No. 39,443; Charles C. 
Cary, Reg. No. 36,764; George A. Willman, 41,378; Steven J. Benerofe, Reg. No. P-41,613; 
David Abraham, Reg. No. 39,554; John Bruckner, Reg. No. 35,816; Travis Dodd, Reg. No. 
P-42,491; U.P. Peter Eng, Reg. No. 39,666; Henry Groth, Reg. No. 39,696; Jinntung Su, Reg. 
No. P-42,174, and to prosecute this application and transact all business in the United States 
Patent and Trademark Office in connection therewith and hereby revokes all prior powers of 
attorney; said appointment to be to the exclusion of the inventors and the inventors' attorneys 
in accordance with the provisions of 37 C.F.R. § 3.71. 
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the Assignee: 

X a copy of an Assignment attached hereto, which Assignment has been (or is herewith) 
forwarded to the Patent and Trademark Office for recording; or 

1 of 2 

::ODMA\PCDOCS\SQLl\227948\l 
Attorney Docket No.: 20308.702 



_ the Assignment recorded on , frames _/_. 

Pursuant to 37 C.F.R. § 3.73(b) the undersigned Assignee hereby states that evidentiary 
documents have been reviewed and hereby certifies that, to the best of ASSIGNEE'S 
knowledge and belief, title is in the identified ASSIGNEE. 

Direct all telephone calls to Kent R. Richardson at (650) 354-4235. 

Address all correspondence to: 

Kent R. Richardson 

WILSON SONSINI GOODRICH & ROSATI 
A Professional Corporation 
650 Page Mill Road 
Palo Alto, CA 94304 

ASSIGNEE: EPIPHANY, INC. 


Name: 


Name: ELIOT L. WEGBREIT 
(Print or Type) 

Title: Chief Executive Officer 
Date: 




::ODMA\PCDOCS\SQLl\227948\l 
Attorney Docket No.: 20308.702 


2 of 2 


TN THE UNITED STATES PATENT AND TRAD EMARK OFFICE 



In re Application ) PATENT APPLICATION 
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Inventor(s): Craig David Weissman. Gregory Vincent ) 

Walsh, Eliot Leonard Wegbreit ) 

) 

) 

Serial No: Not Assigned ) 

) 

Filed: Filed Herewith ) 

) 

T itle: Method and Apparatus for Creating a ) 

Well-Formed Database System Using ) 

a Computer ) 


VERIFIED STATEMENT CLAIMING SMALL ENTITY STATUS 
37 C.F.R. § 1.9(f) AND 1.27(c) - SMALL BUSINESS CONCERN 

I hereby declare that I am: 

The owner of the small business concern identified below. 

X An official of the small business concern empowered to act on behalf of the concern identified 
below. 

Name: Epiphany. Inc. 

Address: 2300 Geng Road. Suite 200. Palo Alto. California 94303 

I hereby declare that the above identified small business concern qualifies as a small business 
concern as defined in 13 C.F.R. § 121.12, and reproduced in 37 C.F.R. § 1.9(d), for purposes of paying 
reduced fees under Section 41(a) and (b) of Title 35 U.S.C. in that the number of employees of the 
concern, including those of its affiliates, does not exceed 500 persons. For purposes of this statement, 
(1) the number of employees of the business concern is the average over the previous fiscal year of the 
concern of the persons employed on a full-time, part-time or temporary basis during each of the pay 
periods of the fiscal year, and (2) concerns are affiliates of each other when either, directly or indirectly, 
one concern controls or has the power to control the other, or a third-party or parties controls or has the 
power to control both. 

I hereby declare that rights under contract or law have been conveyed to and remain with the 
small business concern identified below with regard to the invention. 
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described in: 


X the Specification filed herewith 

Application SC/Serial No. filed 

Patent No. N/A issued N/A 

If the rights held by the above-identified small business concern are not exclusive, each 
individual, concern or organization having rights to the invention is listed below and no rights to the 
invention are held by any person, other than the inventor, who could not qualify as a small business 
concern under 37 C.F.R. § 1.9(d) or by any concern which would not qualify as a small business concern 
under 37 C.F.R. § 1 .9(d) or a nonprofit organization under 37 C.F.R. § 1 .9(e). 

NAME: 

ADDRESS: 

[ ] Individual [ ] Small Business Concern [ ] Nonprofit Organization 

NAME: 

ADDRESS: 

[ ] Individual [ ] Small Business Concern [ ] Nonprofit Organization 

I acknowledge the duty to file, in this application or patent, notification of any change in status 
resulting in loss of entitlement to small entity status prior to paying, or at the time of paying, the earliest 
of the issue fee or any maintenance fee due after the date on which status as a small business entity is no 
longer appropriate. (37 C.F.R. § 1.28(b)). 

I hereby declare that all statements made herein of my own knowledge are true and that all 
statements made on information and belief are believed to be true; and further that these statements were 
made with the knowledge that willful false statements and the like so made are punishable by fine or 
imprisonment, or both, under Section 1001 of Title 18 of the United States Code, and that such willful 
false statements may jeopardize the validity of the application, any patent issuing thereon, or any patent 
to which this verified statement is directed. 

Name of Person Signing: ELIOT L. WEGBREIT 

Title of Person Signing: Chief Executive Officer 


Address of Person Signing: 2300 Gei 
Signature: 

Date: fh. 


d. Suite 200. Palo Alto. California 94303 



" Note: Separate verified statements are required from each named person, concern or 
organization having rights to the invention averring to their status as small entities. (37 C.F.R. § 1.27). 
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Title 37. Code of Federal Regu lations. 8 1.9(c-f) 


(c) An independent inventor as used in 
this chapter means any inventor who (1) has not 
assigned, granted, conveyed, or licensed, and (2) is 
under no obligation under contract or law to 
assign grant, convey, or license, any rights in the 
invention to any person who could not likewise be 
classified as an independent inventor if that person 
had made the invention, or to any concern which 
would not qualify as a small business concern or a 
nonprofit organization under this section. 

(d) A small business concern as 
used in this chapter means any business concern 
as defined by the Small Business Administration 
in 13 CFR 121.12. For the convenience of the 
users of these regulations, that definition states: 

121. 12 Small business for paying reduced 
patent fees, (a) Pursuant to Pub. L. 97-247, a 
small business concern for purposes of paying 
reduced fees under 35 U.S. Code 41 (a) and (b) 
to the Patent and Trademark Office means any 
business concern (1) whose number of 
employees, including those of its affiliates, does 
not exceed 500 persons and (2) which has not 
assigned, granted, conveyed, or licensed, and is 
under no obligation under contract or law to 
assign, grant, convey or license, any rights in the 
invention to any person who could not be 
classified as an independent inventor if that 
person had made the invention, or to any 
concern 


which would not qualify as a small business concern 
or a nonprofit organization under this section. For 
the purpose of this section concerns are affiliates of 
each other when either, directly or indirectly, one 
concern controls or has the power to control the 
other, or a third party or parties controls or has the 
power to control both. The number of employees of 
the business concern is the average over the fiscal 
year of the persons of the fiscal year. Employees are 
those persons employed on a full-time, part-time or 
temporary basis during the previous fiscal year of 
the concern. 

(e) A nonprofit organization as 
used in this chapter means (1) a university or other 
institution of higher education located in any country; 
(2) an organization of the type described in 
section 501(c)(3) of the Internal Revenue Code of 
1954 (26 U.S.C. 501(c)(3)) and exempt from taxation 
under section 501(a) of the Internal Revenue Code (26 
U.S.C. 501(a)); (3) any nonprofit scientific or 
educational organization qualified under a nonprofit 
organization statute of a state of this country 

(35 U.S.C. 201 (i)); or (4) any nonprofit organization 
located in a foreign country which would qualify as a 
nonprofit organization under paragraphs (e)(2) or (3) 
of this section if it were located in this country. 

(f) A small entity as used in this chapter 
means an independent inventor, a small business 
concern or a nonprofit organization. 
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Method and Apparatus for Creating a Well-Formed Database System Using a Computer 

Inventors: Craig David Weissman, Greg Vincent Walsh and Eliot Leonard Wegbreit 

Cross References to Related Applications 

This application relates to the following group of applications. Each application in the group 
relates to, and incorporates by reference, each other application in the group. The invention of 
each application is assigned to the assignee of this invention. The group of applications includes 
the following. 

United States patent application serial number XX/XXX,XXX, entitled “Method and 
Apparatus for Creating a Well-Formed Database System Using a Computer,” filed XX/XX/XX, 
and having inventors Craig David Weissman, Greg Vincent Walsh and Eliot Leonard Wegbreit. 

United States patent application serial number XX/XXX,XXX, entitled “Method and 
Apparatus for Creating and Populating a Datamart,” filed XX/XX/XX, and having inventors 
Craig David Weissman, Greg Vincent Walsh and Lynn Randolph Slater, Jr. 

United States patent application serial number XX/XXX,XXX, entitled “Method and 
Apparatus for Creating Aggregates for Use in a Datamart,” filed XX/XX/XX, and having 
inventors Allon Rauer, Gregory Vincent Walsh, John P. McCaskey, Craig David Weissman and 
Jeremy A. Rassen. 

United States patent application serial number XX/XXX,XXX, entitled “Method and 
Apparatus for Creating a Datamart and for Creating a Query Structure for the Datamart,” filed 
XX/XX/XX, and having inventors Jeremy A. Rassen, Emile Litvak, abhi a. shelat, John P. 
McCaskey and Allon Rauer. 
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Copyright Notice 

A portion of the disclosure of this patent document contains material which is subject to 
copyright protection. The copyright owner has no objection to the facsimile reproduction by any 
one of the patent disclosure, as it appears in the Patent and Trademark Office patent files or 
5 records, but otherwise reserves all copyright rights whatsoever. 

The Field of the Invention 

This invention relates to the field of databases. In particular, the invention relates to creating 
Q databases, and loading and accessing data in the databases. 

• j Background of the Invention 

f: 10 Many different types of databases have been developed. On line transaction processing 

(OLTP) databases are examples of typical databases used today. OLTP databases are concerned 
with the transaction oriented processing of data. On line transaction processing is the process by 
which data is entered and retrieved from these databases. In these transaction-oriented databases, 
every transaction is guaranteed. Thus, at a very low level, the OLTP databases are very good at 
15 determining whether any specific transaction has occurred. 

Another type of database is a data warehouse or datamart. A datamart transforms the raw 
data from the OLTP databases. The transformation supports queries at a much higher level than 
the OLTP atomic transaction queries. A data warehouse or a datamart typically provides not 
only the structure for storing the data extracted from the OLTP databases, but also query analysis 
20 and publication tools. 
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The advantage of datamarts is that users can quickly access data that is important to their 
business decision making. To meet this goal, datamarts should have the following 
characteristics. First, datamarts should be consistent in that they give the same results for the 
same search. The datamart should also be consistent in the use of terms to describe fields in the 
5 datamart. For example, “sales” has a specific definition, that when fetched from a database, 
provides a consistent answer. Datamarts should also be able to separate and combine every 
possible measure in the business. Many of these issues are discussed in the following book, 

Ralph Kimball, The Data Warehouse Toolkit, John Whiley and Sons, Inc., New York, NY 
( 1996 ). 

10 Multi-dimensional datamarts are one kind of datamart. Multi-dimensional datamarts rely on 

a dimension modeling technique to define the schema for the datamart. Dimension modeling 
involves visualizing the data in the datamart as a multi-dimension data space (e.g., image the data 
as a cube). Each dimension of that space corresponds to a different way of looking at the data. 
Each point in the space, defined by the dimensions, contains measurements for a particular 
1 5 combination of dimensions. For example, a three dimensional cube might have product, 

customer, and territory dimensions. Any point in that cube, defined by those three dimensions, 

will represent data that relates those three dimensions. 

The data in the datamart is organized according to a schema. In a dimensional datamart, the 
data is typically organized as a star schema. At the center of a standard star schema is a fact table 
20 that contains measure data. Radiating outward from the fact table, like the points of a star, are 
multiple dimension tables. Dimension tables contain attribute data, such as the names of 
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customers and territories. The fact table is connected, or joined, to each of the dimension tables, 
but the dimension tables are connected only to the fact table. This schema differs from that of 
many conventional relational databases where many tables are joined. The advantage of such a 
schema is that it supports a top down business approach to the definition of the schema. 

Present datamarts have a number of drawbacks that are now discussed. First, datamarts are 
typically difficult to build and maintain. This is because of the requirements that they be 
consistent and flexible. A related drawback of present day datamarts is that they do not allow the 
consultants of the datamart to make changes to the schema simply and easily. Because datamarts 
support very high level queries about the business processes in the business, they require a great 
deal of consistency in the use of data from the OLTP systems. Additionally, the datamarts need 
to be very flexible to address changes in the types of high level queries supported. Changing 
typical datamarts require the changing of hundreds, or potentially thousands, of lines of SQL 
code. For example, if a fact column is added to a fact table, the change propagates throughout 
the datamart. These changes are typically implemented by hand, a very time consuming and 
error prone process. As a result of the hand coding involved, it is quite possible to construct the 
database in an arbitrary fashion that does not conform to good rules for constructing datamarts. 

Thus, well-formed datamarts may not result. 

Thus an improved data warehousing technology is desired. 
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A Summary of the Invention 

One embodiment of the invention includes a method of defining a well-formed database 
system by defining the organization of the data in the database, and by defining the operations 
for that data. The definition can then be used to automatically create and populate the well- 
5 formed database system. The well-formed database system conforms to rules of correctness and 
produces results that conform to the rules. The organization is defined by a data organization 
definition that specifies tables, their columns, and the relationships between tables. The 
operations define procedures that operate on the tables and the table columns. Importantly, the 
5 operations are defined along with the tables, columns, and relationships, so that the resulting 
JO system is well-formed. Without this invention, database systems can be constructed in an 
J arbitrary and inconsistent fashion which can result in an incorrectly constructed database system. 

In some embodiments, when the database system is created, it automatically includes the 
□ following capabilities: foreign key tracking, automatic indexing, time and date information 

^ inclusion. By including some or all of such capabilities in the database system, the system will 

15 operate to comply with the rules of correctness. 

The following are aspects of various embodiments of the invention. The constructed well- 
formed database system can automatically guarantee the following. (1) Two columns related by 
a relational join will be from the same domain. (2) If table A has a many-to-one relationship to 
table B, then table A has a foreign key that corresponds to table B. (3) A many-to-many 
20 relationship, between two tables A and B, is always expressed by an associative table that is 

created in a uniform way. For each unique many-to-many relationship, a unique value is created 

Method and Apparatus for Creating a Well- PATENT Inventors: Craig D. Weissman, 

Formed Database System Using a Computer Page 5 Greg V. Walsh and Eliot L. Wegbreit 

Attorney Docket No 20308 702 
ODMA\PCDOCS\SQL 1 \2280 1 7\ I 



in the associative table and reused whenever that many-to-many relationship occurs. 
Denormalization is always done correctly. (4) Pulling information from one table to be put into 
another table, for access efficiency, is done correctly. 

In some embodiments of the invention, the data organization definition includes a schema 
description for a datamart. The datamart automatically includes the inclusion of transaction type 
information and the mapping of source system keys. In these embodiments, the operation 
definitions define one or more of the following sets of operations: datamart population 
operations, aggregate creation and maintenance operations, query and result interface operations. 

Although many details have been included in the description and the figures, the invention is 
defined by the scope of the claims. Only limitations found in those claims apply to the invention. 
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A Brief Description of the Drawings 

The figures illustrate the invention by way of example, and not limitation. Like references 
indicate similar elements. 

Figure 1 illustrates a datamart system representing one embodiment of the invention. 

Figure 2 illustrates an embodiment of a method of defining the datamart, loading the 
datamart, and then querying the data. 

Figure 3 illustrates a schema used in the system of Figure 1 to define schemas for the 
datamart. 

Figure 4 illustrates a schema used in the data extraction and loading process. 

Figure 5 illustrates a runtime schema including aggregates. 

Figure 6 illustrates a query mechanism and user interface schema. 

Figure 7 through Figure 29 describe a user interface that can be used to define a schema, 

build a datamart, load the datamart, and query the datamart. 

Figure 30 through Figure 36 describe a user interface that can be used by a consultant to set 
up the query interface for a user and to provide the reporting interface. 
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Introduction to the Description 

The following describes a system according to various embodiments of the invention. 
Generally, the system allows a consultant to define a well-formed datamart. The system includes 
tables and columns that conform to the definition of the datamart. The system also includes 
additional columns for foreign key tracking, source system key mapping, time and date tracking. 
The system has automatic indexing. The system enforces typing information about the data 
stored in the datamart. These additional features cause the datamart to operate in a consistent 
manner. One benefit of such consistent operation is that results are consistent in meaning from 
query to query. 
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Focusing on the datamart creation, the system allows a consultant to build a datamart from a 
schema definition and a definition of the sources of the data. From the schema definition, the 
system automatically builds the tables needed in the datamart. Also, from the schema definition, 
and the sources definition, the system can automatically extract the data from those sources. 
Depending on the semantic meaning of the data, as defined by the schema definition, the system 
automatically converts the data from the sources into forms that are readily usable in the 
datamart. Once the datamart has been created, and the data has been loaded, users can then 
perform queries on the data. 

As part of the datamart creation, the system allows the consultant to define aggregates for the 
datamart. The aggregates correspond to pre-computed query results for different types of 
queries. For example, an aggregate can be created for a query that asks for all sales, by region, 
by quarter. The corresponding aggregate table would include a set of rows that have the results 
for this query (e.g., each row includes the quarterly sales for each region). The aggregates are 
specified using the schema definition. This makes defining and changing aggregates relatively 
simple. 

To allow a user to query the datamart, the system includes an interface for defining what 
fields can be used by the user to query the datamart. Additionally, by allowing the consultant to 
define measure and related information, the system allows the consultant to specify how the 
results are to appear to the users. 

The following description first presents a system level view of primarily one embodiment. 
Then, an example use of the system is presented. Next, the metadata used in the system is 
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described. This metadata description is broken into four parts: a top level description of the 
metadata used in defining schemas, a description of the metadata used during the extraction, a 
description of the metadata used while the datamart is running, and a description of the query 
interface metadata. Next, an example set of user interface screen shots illustrates how 
consultants can quickly and efficiently define schemas, aggregates, and query interfaces, and 
how users can query the datamart. Next, additional alternative embodiments are described. 

Definitions 

Datamart or Data Warehouse — is a database. 

Schema — is a description of the organization of data in a database. Often, the schema is 
defined using a data definition language provided by a database management system. More 
abstractly, the schema can be the logical definition of a data model for use in a database. 

Metadata - is data that defines other data. This is not the actual data in the datamart, but is 
the data that defines the data in the datamart. 

Constellation - a grouping of dimension definitions, fact definitions, like-structured facts (all 
facts in a constellation have the same dimensional foreign keys), or stars, and other metadata 
definitions. Often the grouping relates to a business process (e.g., sales). 

Fact Table - the central table of a star schema. It stores the numeric measurements of the 
business that is supplying the information to the datamart. 

Measurement — is a piece of data in a fact table, or an arithmetic combination of data. 
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Dimension - the tables that link to the fact table in a star schema. The tables store the 
descriptions of the dimensions of the business. Examples of dimensions are product and 
territory. 

Attributes - are the fields of a dimension table (e.g., product name, country name). 

User - any end user who would normally wish to query a datamart, but would not usually be 
concerned with the implementation or maintenance of the datamart. 

Consultant - is a person responsible for the creation and maintenance of a datamart. 

Source System - is any computer system that holds the raw data used by the system. 
Examples of such source systems are OLTP database systems. 

Data Store - any data storage (physical or logical) from which data is received or to which 
data is stored. Examples of a data store are files, a database, etc. 

Computer - is any computing device (e.g., PC compatible computer, Unix workstation, etc.). 
Generally, a computer includes a processor and a memory. A computer can include a network of 
computers. 

Program - a sequence of instructions that can be executed by a computer. A program can 
include other programs. A program can include only one instruction. 

Datamart System 

Figure 1 illustrates a datamart system representing one embodiment of the invention. The 
system supports the creation of a well-formed datamart. This system allows consultants to use 
metadata to define schemas for a datamart. From the definition of the schema, the system can 
automatically generate the tables in the datamart. Further, the system can automatically extract 
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the data from the source systems, perform conversions on that data and populate the datamart. 
The system supports the automatic creation and processing of aggregates from aggregate 
definitions. The system also supports the creation of the query mechanisms from query 
definitons. 

5 The following description first lists all the elements of Figure 1, then describes each of those 
elements, and then discusses how those elements operate together. 

System Element List 

Figure 1 includes the following elements: source systems 1 10, a system 100, a web server 
186, a consultant computer 190, and a user computer 180. The system 100 includes the metadata 
10 160, an enterprise manager 102, an extraction program 120, staging tables 130, a semantic 

template conversion program 140, a datamart 150, an aggregate builder 170, and a query and 
reporting program 104. The metadata 160 includes the following data: schema definitions 161, 
connectors 162 (connectors are also referred to as extractors), semantic definitions 163, source 
system information 164, aggregate information 167, measurement information 168, and 
15 query/reporting information 169. The user computer 180 is shown running a browser 182. The 
browser 1 82 includes a query/results interface 1 84. The consultant computer 1 90 shows the 
enterprise manager interface 192 which shows the metadata organization of the system 100. 
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System Element Descriptions 

The following describes the metadata 160, then the other elements of the system 100, and 
finally, the elements that are external to the system 100. These elements are all described in 
greater detail below. 

5 Metadata Overview 

The metadata 160 includes many different types of data and information. This information 
can be broken down into information related to (1) the definition of the schema for the datamart 
150, (2) the data needed during the extraction from the source systems 110 and loading of the 
datamart 150, and (3) the information used in the querying of the datamart 150 and supplying the 
10 result sets. The relationships between the elements of the metadata 160 are described in greater 
detail below. However, the following provides brief descriptions of these elements. 

The schema definitions 161 hold the definition of the schema for the datamart 150. 

Typically, a consultant, using the consultant computer 190, can interface with the enterprise 
manager 102 to define the schema definition 161 for the datamart 150. In particular, the 
15 consultant can use the enterprise manager interface 192 to define a star schema for the datamart 
1 50. This star schema is organized around the business processes of the business for which the 
datamart is being created. What is important is that the consultant can easily define a schema for 
the datamart 1 50 and that definition is kept in the schema definitions 161 . From the schema 
definitions 161, not only can the tables in the datamart 150 be generated, but also the automatic 
20 extraction and conversion of the data from the source systems 110 can be performed, aggregates 
are set up, and a query mechanism is generated. 
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The connectors 162, the semantic definitions 163, and the source system information 164, are 
all related to the extraction of the data from the source systems 110. The connectors 162 define 
the access routines for extracting the source systems data 1 10. The semantic definitions 163 
define how that extracted data should be converted when it is loaded into the datamart 150. The 
5 semantic definitions 1 63 provide important advantages to the system 1 00. In particular, the 
semantic definitions 163 allow for a simplified definition of the datamart 150, consistent 
meaning of the data in the datamart 1 50, and allow for complex changes to the schema to be 
easily propagated to the datamart 150. The source system information 164 defines how to extract 

the data from the systems 1 10. 

l o The aggregate information 1 67 defines how data in the datamart 1 50 is treated once it is 

extracted. The aggregate information 167 allows for the creation of aggregates. Aggregates are 
aggregations of various fields of data in the datamart 150. Aggregates support more complex 
and powerful queries to be executed on the datamart 1 50. The aggregates also improve the 
performance of the system during the querying process and allow for time navigation of the data 
1 5 in the datamart 1 50. Time navigation is the process of creating backlog result sets by hopping 

through date aggregates from the beginning of time in the datamart 150 to the present. 

The measurement information 168 and the query/ reporting information 169 support the 
querying of the datamart 150. A measure is a piece of numeric data in the datamart 150 that is 
useful to a user. That is, individual fact columns from source systems can be very 
20 implementation specific. These columns may not correspond to what users would prefer to see. 
For example, a user may want to see a net price added with a total cost. However, the fact table 
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may only include the net price or the total cost. The measure information 1 68 allows the 
consultant to define the abstract notion of the calculation associated with the net price added to 
the total cost. 

In some embodiments of the invention, the metadata 160 also includes security information. 

5 The security information defines the level of access for various users to the various tables and 
fields in the datamart 150. This security information automatically restricts access to that data. 

System Overview 

The system 100 can be implemented on a network of computers running Windows NT and 
UNIX. The datamart 150 can be implemented on top of an Oracle (SQL Server, or ODBC) 

10 database. However, this physical structure of the system 100 can be implemented in any number 
of ways, and the invention does not require this specific hardware configuration. 

The enterprise manager 102 is a program that is responsible for supporting the definition of 
the schema, and the creation of the tables in the datamart 150 from the schema definitions 161. 
The enterprise manager 102 also controls the extraction program 120. (In some embodiments, 

15 the extraction program 120 and the semantic template conversion program 140 are included in 
the enterprise manager 102). During the execution of the extraction program 120, the extraction 
program 120, the staging tables 130, the semantic template conversion 140, and the datamart 150 
are all used. The extraction program 120 uses the connectors 162 and the source system 
information 164 to extract the information from the source systems 110. The extracted data is 
20 loaded into the staging tables 130. 
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The staging tables 130 are temporary tables used to hold the source system data before 
performing any semantic conversions on that data. The staging tables 130 also allow for the 
conversion of the source system data prior to moving the data into the datamart 150. 

Once the staging tables 130 have been loaded, the semantic definitions 163 can be accessed 
from the enterprise manager 102 to convert the information in the staging tables 130 to 
predefined data semantics. These predefined data semantics allow for powerful queries, 
consistency in the definition of the meaning of the data in the datamart 150, and allow for 
changes to be made to the schema. Generally, the semantic template conversion 140 takes data 
stored in the staging tables 130, performs a conversion of that data according to a corresponding 
semantic definition (defined in the schema definitions 161), and populates the datamart 150 with 

the converted data. 

Importantly, the predefined data semantics substantially simplify the creation and population 
of the datamart 150. In previous systems, the consultant would have to implement all of the data 
manipulation and population programs by hand. By selecting a particular semantic definition for 
a particular fact, or dimension, in the schema, the consultant has automatically defined the access 
and manipulation for populating programs for that table. Allowing the consultant to select a 
predefined data semantic not only reduces the tedious coding previously required of the 
consultant, but also allows for the automatic insertion of foreign keys, transaction types, date, 
and other information into the schema, and therefore the datamart 150. This additional 
information causes the datamart 150 to be well-formed. 
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The aggregate builder 1 70, as mentioned above, aggregates data in the datamart 1 50 
according to the aggregate information 167 and the schema definitions 161. The results of the 
aggregate builder 170 allow for more powerful and faster queries to be performed on the 
datamart 150. 

The query/reporting program 104 supports the querying of the datamart 150 and presents 
results of those queries. The query and reporting process 104 uses the measurement information 
168 and the query and reporting information 169, in addition to the schema definitions 161, to 
query the datamart 150 and provide that information to the web server 186. The query/reporting 
information 169 includes filters and form definitions. The filters allow the user to filter different 
fields out of the datamart 1 50. The forms allow the users to indicate which fields a user is 
particularly interested in. 

The metadata 160, although including many different types of definitional data, importantly 
includes the schema definition 161 and the semantic definitions 163. The enterprise manager 102 
can use the schema definitions 161 to build the tables in the datamart 150. Through the 
combination of these two pieces of metadata 160, the enterprise manager 102 can take data from 
a source system 110, perform semantic conversions on that data and populate the datamart 150. 
Thus, in some embodiments of the invention, the system includes only the schema definitions 
161 and the semantic definitions 163. 

External Elements 

The source systems 1 10, as defined above, represent large databases from which data for the 
datamart 150 is pulled. Examples of such systems include large on line transaction processing 
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(OLTP) systems. Typically these source systems 1 10 are relational databases having multiple 
tables and relations between those tables. The source systems 1 10 do not generally support 
powerful queries that provide high level information about the business in which the source 
systems 1 10 are used. Thus, the system 100 is used to extract the data from the source systems 
110 and to provide an improved schema for querying that data. In some embodiments, the 
source systems 110 include non-relational databases such as object databases. In other 
embodiments, the source systems 110 can include flat file systems or combined relational and 
object databases. What is important is the source systems 1 10 can provide data to the system 
100 through the connectors 162 and the source system information 164. 

The consultant computer 190 represents any computing device that allows a consultant to 
access the system 100. (Access to the system 100 can be through a network.) What is important 
is that the consultant computer 190 allows the consultant to interface with the enterprise manager 
102. Note, that the enterprise manager 102 can run on the consultant computer 190. 

The user can access the web server 1 86 through the user computer 180. In this example, the 
user computer 180 can access the web server 186 through an HTTP connection. The user 
computer 1 80 sends a file request to the web server 1 86. This file request represents a request for 
a query of the datamart 150. The web server 186 runs a Java program upon receiving the 
request. The query and reporting program 104 converts the information from the Java program 
into a query that the datamart 150 will understand. In one embodiment, the query/reporting 
program 104 converts the query into a set of SQL statements. The SQL statements are run 
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against the datamart 150. The results of the statements are processed and provided back to the 


user computer 180. 


Example Method of Defining and Using the Datamart 

Figure 2 illustrates an embodiment of a method of defining the datamart 1 50, loading the 
5 datamart 150, and then accessing the data in the datamart 150. This example can be broken into 
four subparts: a build datamart process 202, an extraction and loading process 204, a build 
aggregates process 205, and a query and reporting process 206. This example can be 
j implemented using the system 100. 

□ At block 2 1 0, a consultant uses the enterprise manager 1 02 to define the schema. The 
lb schema is defined using the metadata 160. This process is illustrated in greater detail in Figure 7 
£ through Figure 35. Generally, defining the schema involves determining the business processes 
3 of the organization for which the system 100 is being implemented. The consultant then defines 

S the star schema for those business processes. The star schema has a fact table and a number of 

S dimensions. The consultant also defines from where the data in the schema is to be derived. 

1 5 That is, the consultant defines from which fields and tables the information is to be extracted 

from the source systems 1 1 0. The consultant also defines how that data is to be put into the 
datamart 150. That is, the consultant associates each piece of data with a semantic meaning. 

This semantic meaning defines how the data from the source system is to be manipulated and 
how it is to populate the datamart 150. At this point, the consultant can also define the 
20 aggregates that can be used in the datamart 150. 
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Once the datamart 150 has been defined, it can then be automatically built. At block 220, the 


enterprise manager 1 02 generates table creation SQL statements according to the definition of the 
metadata. In one embodiment of the invention, block 220 is accomplished by performing queries 
on the schema definitions 161 to generate the fact table creation statements, the fact staging table 
5 creation statements, the dimension table creation statements, the dimension staging table creation 
statements, and the dimension mapping table creation statements. These tables are described in 
greater detail below. From the results of these queries, SQL CREATE TABLE statements are 
created. Importantly, the schema definitions 161 provide the information the enterprise manager 
>0 1 02 needs to build the datamart 150. 

j d Note that this process can also be used to modify the schema of an existing datamart 1 50. 
jg Therefore, at block 220, the SQL tables being created will cause the existing datamart 150 to be 
modified without losing the data in the datamart 1 50. 

£j At block 230, the enterprise manager 102 issues the table generation statements to the 
t; database upon which the datamart 150 is being created. That database creates the tables, which 
15 correspond to the datamart 150. After block 230, the build the datamart process 202 is complete. 

Now the extraction process 204 can be performed. The extraction process 204 is run on a 
periodic basis to load data from the source systems 110 into the datamart 150. This process can 
be run multiple times for the datamart 150. 

At block 260, the connectors 162 are used by the enterprise manager 102, and in particular, 

20 they are used by the extraction program 120 to extract the data from the source systems 1 1 0. The 
connectors 1 62 can include SQL statement templates (not to be confused with semantic 
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templates, as described below) for extracting data from the source systems 1 10. The extraction 
program 120 uses these templates, in addition to the source system information 164, to generate 
SQL statements. These SQL statements are issued to the source system 110 and the results are 
loaded into the staging tables 130. (The staging tables 130 had been created as a result of block 
5 230.) Once the staging tables have been loaded, the data can then be moved into the datamart 

150. 

At block 270, the staging table data is moved into the datamart 1 50 using the semantic 
definitions 163. The semantic definitions 163 are templates for converting the staging tables 130 
data according to predefined data semantics. These predefined data semantics, as described 
lft below, provide semantic meaning to the data being loaded from the staging tables 130. Note that 

:£! the data from the staging tables 1 30, as processed by the semantic template conversion 140, is 

^ placed in the tables in the datamart 1 50. 

Thus, the schema definition and the semantic definitions 163 are used to generate and 
a populate the datamart 150 such that the datamart 1 50 is well-formed. Examples of the well- 
15 formedness of the datamart 150 are as follows. (1) Two columns related by a relational join will 
be from the same domain. (2) If table A has a many-to-one relationship to table B, then table A 
has a foreign key that corresponds to table B. (3) A many-to-many relationship, between two 
tables A and B, is always expressed by an associative table that is created in a uniform way. For 
each unique many-to-many relationship, a unique value is created in the associative table and 
20 reused whenever that many-to-many relationship occurs. Denormalization is always done 
correctly. (4) Pulling information from one table to be put into another table, for access 
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efficiency, is done correctly. Previous systems cannot guarantee such a well-formed database 


system because hand coding of the creation and population operations is required. This hand 
coding can easily introduce errors into datamart creation and population processes. 

Once the extraction process 204 has completed, the aggregates can be built in the build 
5 aggregates process 205. The aggregates are tables of pre-calculated combinations of dimensions 
and facts. Importantly, they greatly increase the speed of queries. Generally, the aggregate 
definitions, stored in the aggregate information 167, are accessed and built using the aggregate 
definitions (which interface with the schema definitions). At block 275, the aggregate builder 
5 1 70 accesses the metadata 160 to build the aggregates. Often, the aggregate building is done at 

10; night. 

Z After aggregates are built, the querying and reporting process 206 can be performed. The 
querying and reporting process 206 can be performed anytime after the creation of the datamart 
1 50. Importantly, when an aggregate is created, the appropriate operation for that aggregate is 
% used. For example, revenue elements are added to produce an aggregate, while daily account 
15 balances are averaged to produce an aggregate. 

At block 277, the consultant defines the query mechanism schema for the system 1 00. In 
particular, the consultant defines the query/reporting information 169 and the measurement 
information 168. These two pieces of metadata 160 allow the system 100 to report meaningfully 
consistent information to users. Also, the consultant is not burdened with having to hand create 
20 the possible reports. 
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At block 280, a query is generated. In one embodiment of the invention the query is 
generated at the query/reporting program 104. In other embodiments, the query can be generated 
at the user computer 1 80 through the HTTP, web server 1 86, Java coupling to the 
query/reporting program 104. What is important here is that some query is generated that can be 
5 used to access the datamart 150. Importantly because the schema definitions 161 are available to 
the query and reporting program 104, the user can be presented with forms from which a query 
can be easily and automatically generated. 

At block 290, the answer set (the results) is created by the datamart 1 50. This answer set is 
then propagated back through the query/reporting program 104, and ultimately to the user 
1 0 computer 1 80. The results are formatted according to the query/reporting information 1 69. 

Ton Level Metadata Schema 

As noted in the background, multi-dimensional datamarts use star schemas. The system 100 
uses star schemas in a larger organization that allows for the sharing of dimension tables by sets 
of similar facts. This larger organization is called a constellation. Figure 3 illustrates a schema 
15 for the schema definitions tables that support constellations. (The schema of Figure 3 is labeled 
the schema for schema definitions 300.) That is, Figure 3 illustrates a schema used in the system 
100 to define schemas for the datamart. Figure 3 also illustrates some of the aggregate 
information 1 67 schema. 

The following describes the meaning of the various graphical elements in Figure 3 through 
20 Figure 5. Each box in the figure represents a table having one or more attributes. A first table 
having a diamond graphic extending to second table (with a dot on the end) indicates that that 
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second table has a foreign key pointing to the first table. This can be thought of as a parent child 
relationship. 

It is important to remember that Figure 3 through Figure 5 illustrate the schema of the system 
used to generate and run the datamart 150. Rows in these tables define the schema for use in the 
5 datamart 1 50. From these rows, create table, table query, etc., commands are created. These 
commands are used to create the tables in the datamart 1 50 and to access that datamart. 

Also, as mentioned previously, the datamart 150 is well-formed because, among other 
reasons, the system 100 automatically includes additional columns in the table created in the 
datamart 150. For example, source system key, foreign key, and time and date columns are 
10 automatically added (where appropriate). The rest of the elements of the system can then rely on 
the existence of these columns. This prevents, for example, the creation of an inconsistent 
schema where only some of the tables include date and time information. 

The following first lists all of the elements in Figure 3 and then describes those elements and 
their relationships. 

15 Top Level Metadata List 

Figure 3 includes the following elements: a constellation 302, a fact table 304, a dimension 
base 306, a semantic instance 308, a fact column 310, a fact aggregate operator 312, a fact 
column number 314, a fact dimension cleansing 316, a dimension role 320, a degenerative 
number 322, a dimension role number 324, a dimension node 326, a dimension column 329, a 
20 dimension column number 32 1 , a cleanse type 323, a cleanse map definition 327, a cleanse map 
325, a physical type 330, a transaction string 332, a metacolumn 334, an actual table type 336, a 
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dimension base type 328, a special dimension base 391, an aggregate key operator 392, an 
aggregate dimension type 393, an aggregate dimension 344, an aggregate group 342, an 
aggregate fact 340, a aggregate dimension set 372, a dimension column set 370, a dimension 
column set definition 374, a fact index 380, a fact index definition 384, and a fact index number 
5 382. 

Top Level Metadata Descriptions 

It is important to remember that the tables in Figure 3 are only used to define the schema in 
the datamart 1 50. Thus, a fact table 304 in Figure 3 is not the actual fact table in the datamart 
5 150, but the definition of that fact table. Each row in a table corresponds to an instance of that 

Ao table. 

% The constellation 302 defines the organization of the schema in the datamart 1 50. It is the 

□ top level table in the schema definition. 

=; Fact Related Tables 

IS The fact table 304 defines the metadata 1 60 table describing all of the fact tables within a 

15 given constellation 302. The attributes of the fact table 304 include a build aggregates flag, a 
cleanse flag, a constellation key, a description, a fact table key, a fact table name, and a truncate 
stage flag. Each attribute corresponds to a column in the fact table 304. The build aggregate flag 
indicates whether or not to build aggregates for a particular fact on the next execution of the 
aggregate builder 170. The cleanse flag is a flag that is used in many of the tables to obliterate 
20 the actual measures within a table in the datamart 150 (particularly useful in demonstrations of 
the system 100 where sensitive data would otherwise be revealed). The constellation key points 
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to the parent constellation 302 for a given fact table 304. The fact table name is the name of the 
fact table used in constructing the corresponding physical table names in the datamart 150. The 
truncate stage flag is used to indicate whether or not to truncate the fact staging table on the next 
extraction. 

5 The fact column 310 lists all of the fact attributes within a single fact table 304. The fact 
column 310 includes a cleanse flag, a description, a fact aggregate operator, a fact column key, a 
fact column name, a fact column number, a fact table key, and a physical type. The fact 
aggregate operator is an SQL operator used to aggregate this fact column in the datamart 1 50. 

% The fact column key is the primary key for the fact column. The fact column name is the 

~I 10 physical name of the fact column. The fact column number counts and orders the number of 

! columns in the fact table. The fact table key points to the fact table to which the corresponding 

® fact column belongs. The fact table key points to the fact table to which the fact column belongs, 

i fl The physical type is the database type for the fact column. This type is a logical type and 

ITi provides for independence of implementation of the datamart 1 50 from the underlying database 

•-- ; 15 used. 

The fact column number 314 and the fact aggregate operator 312 are used by the fact column 
3 1 0. These have already been described in the context of the fact column 3 1 0. 

The fact dimension cleanse table 316 has rows that indicate the dimension foreign keys in a 
fact that should be cleansed. The fact dimension cleanse table 3 1 6 includes a dimension role 
20 key, a fact dimension cleanse key, and a fact table key. The dimension role key indicates that 
this dimension role 320 is part of the “group by” set for cleansing a fact table without distorting 
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trends in the data. The fact dimension cleanse key is the primary key for the fact dimension 
cleanse table 316. The fact table key is the fact table having its cleansing properties. 


Dimension Related Tables 

The dimension base 306 is the metadata 160 describing all the dimension tables that can be 
5 used in a given constellation 302. These dimension bases can then be used in multiple 

constellations. The dimension base 306 includes the following attributes: an aggregate key 
operator, a cleanse flag, a description, a dimension base key, dimension base name, a dimension 
base type, and a truncate stage flag. The aggregate key operator is an SQL operator used by the 
aggregate builder 170 to build aggregates from a dimension. The cleanse flag and description act 
10 similarly to those attributes in other tables. The dimension base key is the primary key for the 
dimension base 306. The dimension base name is the name of the base dimension used in 
constructing real tables in the datamart 150. The dimension base type indicates the type of a 
dimension base (either default or special (special includes “date” and “transaction type,” which 
are used by the system 100). The truncate stage flag operates in the manner similar to other 
15 truncate stage flags. 

The dimension column 329 defines the list of dimension attributes that are valid for a single 
base dimension 306 and inherited by a dimension usage. The dimension column 329 includes a 
cleanse label, a cleanse map key, a cleanse type, a description, a dimension base key, a 
dimension column key, a dimension column name, a dimension column number, a dimension 
20 number key, grouped by field, a physical type, a primary key, a time navigation field, and a 

default value. The cleanse label is a label presented to users after this column has been cleansed. 
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The cleanse map key is for use when cleansing using value mapping. The cleanse map key 
indicates the mapping group to use. The cleanse type is the method for cleansing the dimension 
column 329. The description is for documenting the dimension column 329. The dimension 
base key is the numbered base in which the column resides. The dimension column key is the 
5 primary key for the dimension column 329. The dimension column name is the physical name of 
the column. The dimension column number is the count of the dimension columns (to prevent 
too many from being created in the datamart 150). The dimension node key is the aggregate 
hierarchy group in which the column resides. The “group by” field is used for special 
dimensions to indicate whether or not this column needs to be “grouped by” during the 
10 processing by the aggregate builder 170. The physical type is a logical database type for this 

dimension column 329. The primary key is used in special dimensions to indicate whether or not 
this column is the primary key. The time navigation field is for the date special dimension to 
indicate whether or not time navigation should use this field. The default value is the default 
value for the dimension column. 

15 The dimension column number 321 is a look up table for the valid number of dimension 
columns that can be created. The dimension column number counts the number of dimension 
columns to make sure there are not too many being defined by the consultant. 

The dimension role 320 is a metadata 1 60 table that describes all of the dimension tables 
used in a constellation 302. The dimension role 320 includes a constellation key, a degenerative 
20 number, a description, a dimension base key, a dimension role key, a dimension role name, and a 
dimension role number. The constellation key points to the constellation 302 in which the 
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dimension role 320 resides. The degenerative number defines the order of degenerate columns 
within fact tables in a constellation. The description is a documentation field for describing a 
dimension role. The dimension base key is the dimension base that this dimension role refers to. 
The dimension role key is the primary key for the dimension role 320. The dimension role name 
5 is the name of the dimension role and is used when constructing the foreign keys in the fact 
tables in the datamart 150. The dimension role number defines the order of the dimension roles 
within a constellation. That is, a constellation may have multiple dimension roles and the 
dimension role number allows for an ordering of those dimension roles. 

;j The dimension node 326 is a table used for defining and grouping hierarchical dimension 

io attributes that are used by the aggregate builder 170. The dimension node includes a dimension 
base key, a dimension note key, a node name, a node number, and a parent node number. The 
^ dimension base key points to the dimension base being defined. The dimension node key is the 
~ primary key for the dimension node. The node name is the logical name for the aggregate 

?! hierarchy group being defined. The node number is the logical number for the aggregate 

B 15 hierarchy group being defined. The parent node number is a logical number for the parent of the 
aggregate hierarchy group being defined. 

The degenerative number 322 and the dimension role number are defined as described in the 
dimension role 320. 

The dimension base type 328 is defined as described in relation to the dimension base 306. 
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Semantic Instance Table 

The semantic instance 308 is a single record that represents the manner in which a fact or 
dimension table is extracted from staging tables, manipulated, and then used to populate the 
corresponding table in the datamart 1 50. The semantic instance 308 includes an extraction node 
5 key, dimension base key, a fact table key, a semantic instance key, and a semantic type key. The 
extraction node key points to the extraction node that a particular semantic instance belongs to. 
The dimension base key is the dimension base table owning this semantic instance. The fact 
table key points to the fact table owning this semantic instance. Only one of the dimension base 
j key and the fact table key is filled in for a semantic instance 308 because the semantic instance 

1 10 can only be applied to one or the other. The semantic instance key is a primary key for the 

j semantic instance 308. The semantic type key is the indicator of the type of transformation 
y necessary to construct this type of semantic instance in the datamart 150. 



The aggregate group 342 includes an aggregate group key, an aggregate group name, a 
constellation key, a default flag, a description, and an enabled field. The aggregate group key is 
the primary key for the aggregate group 342. The aggregate group name is the logical name of 
this aggregate group. A constellation key points to the constellation in which this aggregate 
5 group resides. The default flag indicates whether or not this group is the default group within a 
constellation. Default groups have facts and dimensions automatically added to them. They can 
also not be deleted. The description contains the documentation for this aggregate group. The 
enabled field indicates whether or not the aggregate builder 170 will actually build this group. 

The aggregate fact table 340 tracks the membership of a fact within an aggregate group. The 
10 aggregate fact table 340 includes an aggregate fact key, an aggregate group key, and a fact table 
key. The aggregate fact key is the primary key for the aggregate fact 340. The aggregate group 
key is the aggregate group being defined by the aggregate fact. The fact table key points to the 
fact table that is being made a member of the group. 

The aggregate dimension 344 indicates the membership of a dimension within a constellation 
15 in an aggregate group. The aggregate dimension 344 includes an aggregate dimension key, an 
aggregate dimension type, an aggregate group key, a dimension role key, and a special 
dimension base key. The aggregate dimension key is the primary key for the aggregate 
dimension 344. The aggregate dimension type indicates the manner in which this dimension 
(special or role) will be included in the aggregate group. The aggregate group key indicates the 
20 aggregate group being defined. The dimension role key points to the dimension role being 
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included. It is possible that this key is null. The special dimension base key indicates the special 
dimension being included. The special dimension base key can also be null. 

The aggregate key operator 392 is defined as described in the dimension base 306. 

The fact aggregate operator 3 12 is a look up table of valid fact aggregation operations. Each 
5 operator is an SQL operator used to aggregate a fact column. 

A special dimension base 391 provides details about special built-in dimensions in the 
system 100. The special dimension base includes an “always include an aggregate” field, a 
default aggregate dimension type, a dimension base key, a list order in fact, a physical type of 
key, an index flag, and a special dimension base key. The “always include an aggregate” field 
10 indicates whether or not this dimension table must always be included in all aggregates. The 
default aggregate dimension type is the default manner in which this dimension is included in 
aggregate groups. The dimension base key is the one to one relationship to a dimension base. 
The list order in fact is the order in fact tables that the foreign key to this table will be listed. The 
physical type of key is the logical database type that foreign keys in the fact tables that point to 
15 this special dimension will be. The index flag is used in indexing. The special dimension base 
key is the primary key for the special dimension base 391 . 

Data Store Related Tables 

The physical type 330 defines a look up table of logical data types that are relational database 
management system (RDBMS) independent. The physical type 330 is a logical data type that 
20 works across various source systems storage types. The physical type 330 includes a database 
physical type, a default value, and a special type. 
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The translation string 332 defines a list of strings that are translated for different RDBMS's. 
The translation string is a logical string that can be converted to specific strings for different 
storage types. Each storage type would correspond to a different source system 1 10. 

Cleansing Related Tables. 

5 The cleanse type 323 is a look up table to indicate how to cleanse a dimension column. 

The cleanse map 325 is a mapping table for mapping real names to cleanse names. The 
cleanse map 325 includes a cleanse map key, which is the primary key, and a cleanse map name, 
which is the name of a set of mapping pairs for the purpose of scrambling data. 

The cleanse map definition 327 defines the details of what should be mapped to which fields. 
10 The cleanse map definition 327 includes cleanse map character ID, a cleanse integer ID, a 

cleanse map definition key, a cleanse map key, and a cleanse value. The cleanse character ID is 
a character value for indexing into this mapping group. The cleanse integer ID is a numeric 
value for indexing into this mapping group. The cleanse map definition key is the primary key 
for the cleanse map definition. The cleanse map key is the mapping set to which this particular 
15 cleanse map definition belongs. And the cleanse map value is the translation value after the 
mapping. 

Additional Tables 

The metacolumn 334 is a column that occurs by default in tables in the datamart 1 50. The 
metacolumn 334 includes an actual table type, a list order, a metacolumn key, a metacolumn 
20 name, and a physical type. The actual table type indicates the type of physical table in which this 
special column should appear. The list order is the order this column occurs in tables of the 
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appropriate type. The metacolumn key is the primary key. The metacolumn name is the 
physical name of the column when it is used. The physical type is the logical data type for this 
column. 

The actual table type 336 is a look up table for actual table types. Actual table types can be 
5 fact, dimension stage, fact stage, dimension map, or dimension. 

The aggregate group 342, the aggregate fact 340, the aggregate dimension set 372, the 
dimension column set 370, the dimension column set definition 374, the fact index 380, the fact 
index definition 384, and the fact index number 382 are for future use and are therefore optional. 
■=; Each of these tables provides greater flexibility when defining the metadata 1 60, improves the 
it) performance of the system 100, or may otherwise enhances the system 100. 

Z: Top Level Metadata Use 

= : Now that all of the elements in Figure 3 have been listed and described, their relationships 

;j and workings are now described. 

::0 It is important to note that many of the tables in Figure 3 are actually used in providing layers 

15 of abstraction to allow for the reuse of information and non-abstract tables. Therefore, a 

consultant will often only deal with only some of the tables in the Figure 3. For the purposes of 
describing how the metadata 160 can be used to define a schema for the datamart 1 50, these 
grouping and levels of abstraction tables will be described where appropriate. 

Generally, a consultant will create a new datamart 150 by defining instances of the dimension 
20 bases 306, and constellations 302. Each instance corresponds to a row in the dimensions bases 
306 table or the constellation 302 table. The constellation instances are defined by defining 
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aggregates, dimensions, facts, measures, and ticksheets. The following describes the definition 
of a schema using the metadata 160. This corresponds to block 210 of Figure 2. 

Beginning with the facts in a constellation, the consultant defines a fact table 304 row that 
will define the hub table in a star schema supported by the constellation. Again, it is important to 
5 remember that the fact tables in Figure 3 are for definitional purposes, and are not the real fact 
tables in the datamart 150. A row in the fact column 310 holds the details of what columns will 
be created for place holders of actual values in a corresponding fact table. Thus, for each fact, 
the consultant defines the various fact columns. 

Once the facts have been defined, the consultant can then define the dimensions of the 

10 constellation. 

Si Remember that the dimension base 306 holds the information to define the actual dimensions 
of the tables in the datamart 150. The dimension role 320 allows for the reuse of the dimension 
;Tj base tables. Thus, different dimension roles can refer to the same dimension base. This provides 
m an important feature of some embodiments of the invention where the same dimension bases can 

11 be used in multiple constellations or within the same constellation. The dimension columns 329 
define the columns on which queries can be performed in the datamart 150. The dimension node 
table 326 helps relate the dimension columns 329. Thus, the consultant will have defined the 
basic schema for the datamart 150. 

The aggregate group 342 defines how particular facts or dimensions are to be aggregated by 
20 the aggregate builder 170. These aggregated facts provide much faster queries in the datamart 
150. 
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The cleansing map tables are for scrambling the data in the datamart 1 50 for presentations to 
people who want to see the functionality of the system 100, without having to reveal the actual 
data in the datamart 150. 

The special dimensions are the transaction type table and date values that are included in 
5 every fact table. Because this is included in every fact table, the system 1 00 can rely on the 

existence of the transaction type during the various stages of datamart 150 creation, modification, 
querying, and the like. 

Thus, the elements of Figure 3 can be used to allow the consultant to define the schema 
definitions 161 for creating the tables in the datamart 1 50. 

■ ; l 

ldj Extraction Metadata 

;|r The following describes the metadata 160 used in the extraction process 204. This metadata, 
0 represented as extraction schema 400, is shown in Figure 4. The extraction process focuses 

□ around the job and connector tables. In general, these tables define the various steps in 

y extracting the source system data into the staging tables 130 and performing the desired semantic 
15 conversions on that data. 

Extraction Metadata List 

Figure 4 includes the following elements: a job 402, a job step 404, a system call 405, a 
connector 406, a connector time stamp 407, a connector step 408, a connector column latch 409, 
and an extraction group 41 1, an extraction note 410, an SQL statement 420, and error handling 
20 type 413, an external table 422, an external column 424, the physical type 330, the fact table 304, 
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a debug level 415, the semantic instance 308, a semantic type 430, a dimension semantic type 
432, a fact semantic type 434, the actual table type 336, a semantic type definition 436, an 
adaptive template 438, and adaptive template block 439, the dimension base 306, a job log 401, a 
connector store role 448, a store role 446, a statement type enabled 428, a store role allow 444, a 
5 data store 440, a source system 442, a file store 441, and Oracle store 454, a store version 452, 
and SQL server store 456, and ODBC store 458, and a store type 450. 

Extraction Metadata Descriptions 
Job Related Tables 

; The job 402 is a top level object for controlling the work flow during the extraction and 

lift! loading process 204. The job 402 includes a check databases field, a check tables field, a 

J; description, a label, an initial load flag, a job key, a job name, a log file width, a mail to on error, 

a mail to on success, and a truncate flag. The check databases field indicates whether or not an 
□ attempt should be made to log into all the data stores before executing the job. The check tables 

■V flag indicates whether or not to check for the existence of all the tables in the datamart 150 

15 before executing the job. The description is for documenting the job (usually done by the 
consultant). The enabled flag indicates whether or not a particular job can be run. The initial 
load flag indicates whether or not to ignore all previous time stamped constraints when running a 
particular job. The job key is the primary key for the job table 402. The job name is the internal 
name of the job. The log file width indicates how many characters wide to make rows in the log 
20 file output. The mail to on error, and the mail to on success indicate where E-mail messages 
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should be sent after failure or success of the particular job. The truncate flag indicates whether 
or not to truncate any tables when running a job. 


The job log 401 is the locations where the running job is logged. That is, the location of the 
output that will be provided to the consultant indicating what occurred during the extraction (e.g., 
5 what errors occurred). The job log 401 includes a data store key, a job key, a job log key, and a 
job store role. The data store key indicates the data store having a role defined within the job. 
The job key is a reference to the particular job, the job log key is the primary key for the job log 
401 . The job store role is the role being assigned to that particular job log 401 . An example job 
is store role is “<working directory>,” indicating the path to the working director where job log 
re; files are store. 

: \i The job step table 404 includes the detailed steps that make up a job. This includes 

~ connectors and system calls. The job step table 404 includes the following attributes: a 

I S connector key, a description, an enabled flag, a job key, a job step key, a list order, a phase, a job 

m step type, and a system call key. The connector key indicates the connector being included in 

15- any particular job step. The connector key can be null. The description is for documenting the 

job step. The enabled flag indicates whether or not a particular step is enabled. The job key 
points to the job being defined by the job step 404. The job step key is the primary key. The list 
order indicates the order of a particular job step. The phase also indicates the order of a 
particular step. By supporting both list order and phase, alternative embodiments of the 
20 invention can support parallel extraction. Steps in the same phase can then be executed 

simultaneously. The job step type indicates the type of the job step (which are defined in the job 
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step type table 481). The system call key points to a system call included in a particular job step. 
The system call key can be null. 

The system call 405 is a table including external OS system calls. These external system 
calls can be used to perform any number of external system functions. The system call 405 
5 includes the system call key, a command string, a description, a name, and an on-error type. The 
system call key is the primary key for the system call. The command string is the actual 
operating system command to be run as a result of the system call. The description is for 
documenting the system call. The name is the logical name of the system call being defined, 
i The on error type is an indicator to point to what to do if the system call fails. 

lQj Connector Related Tables 

^ The connector 406 defines a name and a description. The connector 406 is a grouping 
mechanism for extraction statements and a specification for input and output data stores. The 
/} description is used for documenting the connector. The name is the logical name of the 
= n connector. The connector 406 represents an ordered collection of connector steps 408. 
if The connector step 408 defines steps within a connector. The connector step table 408 
includes the following attributes: a connector key, a connector step key, an enabled flag, an 
extraction node key, a list order, and a phase. The connector key points to the connector being 
defined. The connector step key is the primary key. The enabled flag indicates whether a 
particular connector step is enabled. The extraction node key points to an extraction node that is, 
20 the extraction group of statements and semantics that make up this connector step. The list order 
is the order of steps in the connector. The phase is also the order of the steps in the connector. 
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The connector time stamp 407 relates to information about incremental extraction. An 
incremental extraction is where increments of the data in the source system 110 are extracted. 

The connector time stamp includes a connector key, a connector time stamp key, current max 
date, a current max time stamp, a last max date, and a last max time stamp. The connector key 
5 points to the connector to which the connector time stamp applies. The connector time stamp 
key is a primary key. The current max date is an indicator of the proposed new system date of 
the last successful extraction. The current maximum time stamp is the proposed new SQL server 
time stamp field for the last successful extraction. The last maximum date is the system date of 
the last successful extraction. The last maximum time stamp is the SQL server time stamp field 
icy for the source system databases at the last successful extraction. 

4 The connector time stamp 407 is particularly useful when only updated data should be pulled 
' J from the source systems 1 10. 

,i. The connector column latch 409 defines information about incremental extraction based on a 
nr j database column. The incremental extraction information is thus kept in the database and can be 
ii retrieved. The connector column latch 409 includes the following attributes: a column name, a 
connector column latch key, a connector key, a current maximum value, a last maximum value, 
and a table name. The table name is the name in the input data store for the corresponding 
connector. The column name is the column name within that table. The connector column latch 
key is the primary key. The connector key points to the connector to which this latch applies. 

20 The current max value represents the proposed new maximum value for the incremental 
extraction. This number is pushed into the last maximum value if the currently executing 
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extraction succeeds. The last maximum value is the maximum value that was extracted during 
the last run of the extraction. 

The connector store role 448 defines the usage of a data store for a particular connector. It 
indicates whether the data store is input or output. The connector store role points to the 
5 connector and the data store. The connector store role also indicates the type of storage usage 
being defined for this connector (input or output). 

Extraction Group Related Tables 

The extraction group 411 defines a group of extraction steps. The extraction group 411 
includes a description, and a name of the set of extraction steps. 

10 The extraction node 410 is a single node, or step, in the extraction tree. The extraction tree 
defines the order of extraction steps. An extraction node includes an extraction node type, a 
debug level, a debug level row, an enabled flag, an extraction group key, an extraction node key, 
a list order, and on error type, a parent extraction node key, and a phase. The extraction node 
type defines the type of extraction node (as defined in the extraction node type table 491). This 
1 5 relationship is important and allows for the conversion of data in the staging tables for use in the 
datamart 150. The debug level indicates how to debug a particular step during execution. The 
debug level row indicates which row to start debugging at for SQL statements. The enabled flag 
indicates whether or not to execute a particular SQL statement associated with the extraction 
node. The extraction group key points to, if not null, the name of the group. The extraction node 
20 key is a primary key. The list order and the phase define the order of the corresponding step 
within an extraction node’s parent. The on error type indicates what to do if there is an error 
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during the execution of the step associated with the extraction node. The parent extraction node 
key points to the parent extraction node of the present extraction node. 

SOL Statement Related Tables 

The SQL statement 420 defines a single step in an extraction run. A row in the SQL 
5 statement 420 table represents an SQL statement. The columns in the SQL statements match 
those in the corresponding dimension base definition or fact table definition. In one embodiment 
the consultant supplies the SQL source in SQL statements. 

The SQL statement 420 includes an extraction node key, a description, a dimension base key, 
and execute against input flag, an external table key, a fact table key, SQL source and SQL 
10 statement key, and an SQL statement name. The extraction node key points to the extraction 
node associated with a particular SQL statement 420. The description is for documenting the 
SQL. The dimension base key points to the dimension base for the corresponding SQL 
statement. The execution against input flag indicates whether or not to execute this SQL 
statement against the source or destination data store of the connector that is calling this SQL 
1 5 statement. The external table key points to the external table, if any, being extracted into. The 

fact table key points to the fact table, if any being extracted into. The SQL source is the actual 
SQL source to be executed during the SQL statement execution. The SQL statement key is the 
primary key. The SQL statement name is the logical name for an extraction SQL statement. 

The statement type enabled 428 defines which RDBMS types use this extraction statement. 

20 The statement type enabled 428 includes the following attributes. An SQL statement key, a 

statement type enabled key, a store role name, and a store type. The SQL statement key points to 
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the SQL statement. The statement type enabled key is the primary key. The store role name is 
the role name for which this SQL statement should be used. The store type is the store type for 
which this statement should be executed. 

The external table 422 defines the user defined destination table for use during a multiphase 
5 extraction. This user defined destination table can be used to temporarily store data during an 
extraction. The external table 422 defines the physical name of the external table and whether or 
not to truncate this external table on the next job run. 

The external column 424 defines a column in a user defined extraction table. The external 
'% column 424 includes the attributes for documenting a particular external column, and the column 

sj 10 name in the external table. A pointer to the external table, a list order of appearance in the 

2 external table, and a physical type of the logical database for this column are included in the 
” external column 424. 

g Error Handling 

g The error handling type 41 3 is a look up table to define how to respond to a particular error. 

"15 An error handling type can be a default action to take when an error occurs. 

The debug level 415 defines ways in which extraction steps can be debugged. The debug 
level includes a logical name for users to pick a debugging behavior. The default level also 
includes a default flag. 

Data Semantic Related Tables 

20 The semantic type 430 defines a set of predetermined semantic types for use in defining a 
schema. The semantic type includes a logical name for a particular transformation. Associated 

Method and Apparatus for Creating a Well- PATENT Inventors: Craig D. Weissman, 

Formed Database System Using a Computer Page 44 Greg V. Walsh and Eliot L. Wegbreit 

Attorney Docket No 20308 702 
ODMA\PCDOCS\SQL 1 \2280 1 7\ 1 



with the semantic type are a dimension semantic type 432 and a fact semantic type 434. The 
dimension semantic type table 432 defines the ways in which dimension data in the staging 
tables 130 can be extracted and put into the datamart 150. Similarly, the fact semantic type 
defines the ways in which the information in the staging tables 130 can be put into the fact tables 
5 of the datamart 150. Both the fact semantic type 434 and the dimension semantic type 432 

include pointers to an actual table type and are used to subset the full list of semantic types. 

Each semantic type 430 is made up of a semantic type definition. The semantic type 
definition table 436 defines the set of adaptive templates used in any given semantic type. The 
semantic type definition 436 includes the semantic type key that points to the semantic type 430 
JO for a particular semantic type definition. The semantic type definition also points to the adaptive 
^ template 438 used. The semantic type definition also includes a list order for the ordering of the 
adaptive templates. 

n The adaptive template 438 is a semantic transformation template (e.g., an SQL program) that 

iP is used in the extraction of the data in the staging tables 130 to turn all source data into 
*15 transactional data. The adaptive template 438 includes attributes indicating a logical name for an 
adaptive program used within semantic transformations. 

The adaptive template block 439 defines the individual pseudo-SQL statements that make up 
a template. The adaptive template block 439 has the following attributes: the adaptive template 
block key, an adaptive template key, a block name, a list order, an on error type, and SQL source. 
20 The adaptive template block key is the primary key. The adaptive template points to the adaptive 
template to which this block belongs. The block name is the internal logical name for this block 
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of pseudo-SQL source code. The list order is the order of this block within the template. The on 
error type indicates what to do if this block causes an SQL error when executed. The SQL 
source includes a template of pseudo SQL source code. This template is described in greater 
detail below. 

5 Data Store Related Tables 

The store type 450 defines the types of RDBMS's supported by the system 100. 

The data store 440 defines a logical data source, or sink, used during the extraction. The data 
store 440 includes the following attributes: the data store key, a data store flag, a description, a 
~ name, a source system key, and a store type. The data store key is the primary key. The 

rjo datamart flag indicates whether or not this data store is the special datamart store. Since the 

datamart 1 50 and the metadata 1 60 can reside in the same database or different, the data mark 
helps resolve the location of the datamart 150. The description is for documentation of the 
n particular data store. The name is the logical name of the data store. The source system key 
Jj points to the source system identifier to which this data store belongs. This allows live, and 
Ms backup, source systems to share the same identifier. The store type indicates the store type of 
this data store. 

The source system 442 is a logical identifier for a source system 1 10 from which data can be 
pulled. This allows two physical databases to act as one master database and one backup, for 
example. The source system 442 includes a description attribute, a source system key and a 
20 source system name. The description is for documentation to describe the source system. The 
source system key is a primary key for this table. This number also becomes identified source 
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system field in the staging tables 130 being filled. The source system names is a logical name 
for a source system 110 from which the system 100 is pulling data. 

The following store tables are subtypes of the data store table 440. They address specific 
data stores. 

5 The file store table 441 defines the files in which data can be stored. The file store 441 
defines a directory and file name for each file. 

The Oracle store 454 defines information about particular Oracle databases. The Oracle store 
454 includes the following attributes: a data store key, an instance name, an Oracle store key, a 
% password, an SQL network name, a user name, and a version. The data store key is a one to one 

Ip relationship key to the data store being defined. The Oracle store key is the primary key. The 

y password and user name are used to access a specific Oracle system. The version number is the 
’ Oracle vendor version number. The SQL network name is the SQL net instance name. The store 

n version is a version of the store type (the database vendor's version for example) that the system 

^ recognizes. The store version has a pointer to the store type being defined and also includes a 
f5 version number attribute. 

The SQL server store table 456 defines details about an SQL server system. The SQL server 
store includes the following attributes: a data store key, a database name, a password, a server, 
and SQL server store key, a user name, and a version. The data store key is a one to one 
relationship to a data store entry. The database name is an SQL server database name 
20 ($$DEFAULT means the database in which this role resides). The password is the SQL server 
password. $$ DEFAULT again means the password currently logged into to read this data. The 
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server is the SQL server name. The SQL server store key is the primary key. The user name is 
the SQL server user name. The version is the vendor's version number of this SQL server. 
$$DEFAULT means use the default value for the current database being used. For example, the 
database name means the database in which this role resides. 

5 Extraction Metadata Use 

The following describes the tables of Figure 4 in the context of the extraction process 204. 
The job, the job step, and the connector, group extraction steps for extracting information from 
_ the source systems 1 1 0 and cause that information to be placed in the datamart 150. This 

5 organization allows for a very flexible extraction process. For example, where a two phase 

ID extraction is required, one connector could be used to extract the information from the source 
tl system 1 1 0, while a second connector could then be used to take this extracted data from an 
external table. 

ill The job defines the order of the execution of the connectors. The job also allows for the 
J: running of an external program, such as system call, between connector executions. Thus each, 

15 job step in a job can be a system call or a connector. 

The following is an example illustrating the organization of job. Assume that a consultant 
wants to extract information from a source system that provided a raw set of home addresses. A 
system call could be run as part of a job step. The system call would determine the zip codes 
associated with those addresses. The zip codes could then be included in the datamart 150. 

20 The relationship between the connector 406, the connector step 408, and the extraction node 

41 0 is that the connector 406 allows for the reuse of extraction nodes in multiple connectors 
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(through the connector step). This relationship is particularly important where similar connectors 
are created. For example, assume that a consultant wants to create a connector that runs some 
steps Monday through Friday and different steps on Saturday and Sunday. Most of the steps in 
the connector will be common, however some will be different. Through the use of the 
connector step, the consultant can reuse many of these connector steps in each connector. 

As noted before an extraction node can be a leaf or a grouping mechanism for extraction. It 
can correspond to an SQL statement extraction step or a semantic conversion step. During the 
definition of the schema, the consultant defines the specific SQL statements for extraction and 
the specific semantic instances for the facts and dimensions. 

An SQL statement is a single step in an extraction run that represents a data push or a data 
pull. The SQL source code dictates the action for a given extraction node. After the SQL 
statements are run, the staging tables 1 30 are ready. The semantic conversion of the data in the 
staging tables 130 can occur. 

The semantic instance represents the use of a single generic template on one fact or 
dimension table. The semantic type associated with the semantic instance is one of a number of 
pre-defined recognized data meanings within the system 100 (e.g., an “order”). The semantic 
types correspond to programs for converting the data in the staging tables 130 into data for use in 
the datamart 1 50. An example of a semantic type is a “slowly changing dimension” type. 

The semantics types, as mention previously, are made up of a series of templates. These 
templates include tokens that can be replaced with information from the corresponding 
dimension base or fact table. An example of an adaptive template is one that would be used in 
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re-indexing of a fact table. This could be used as the last step in the semantic transformation of 
facts. The re-indexing will help speed the operation of the datamart 1 50. Importantly, this same 
indexing is performed for each fact table. No matter which semantic type is chosen for a given 
fact table, the same indexing is performed. Thus, this adaptive template can be used in each 
5 semantic type through its semantic type definition. 

The following describes the slowly changing dimension semantic type. (See Appendix A.) 
In this semantic type are an insert dimension and an index dimension adaptive template. Each 
adaptive template has a corresponding set of pseudo-SQL statements. During the semantic 
;; template conversion 140 this pseudo-SQL will be transformed into real SQL source code. This 
40 is done by converting the pseudo-SQL tokens into actual dimension column names, etc. (the 
2 column names and table names are derived from the schema definitions 161). 
y Thus, during the extraction, the extraction node associated with a particular semantic type 

m instance is processed. This causes the adaptive template blocks associated with the semantic 
o'i instance to be processed. The dimension information associated with that semantic instance, or 

45 the fact table information associated with the semantic instances, can then be used to replace the 

tokens with the specific information associated with that dimension or that fact. 

Some embodiments of the invention correspond to only one or more semantic templates and 
a computer readable media, a computer, an electromagnetic waveform, or the like. 

Further Discussion of Templates 

20 The following describes the pre-parsed template and the post-parsed results in greater detail. 
Each token begins with a $$. In the example template, for the slowly changing dimension 
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semantic type, a number of tokens begin with $$ DIM KEY. Similarly, tokens appear that begin 
with $$FSTGTBL[]. In the post-parsed template, the dimension key tokens have been changed 
to cost center keys, account key, subaccount keys, etc. Note any tokens, and their surrounding 
text, that are not replaced are removed from the post-parsed text. If more tokens need to be 
replaced then are available in the template, then an error flag will be set. In other embodiments 
of the invention, the templates are dynamically generated according to the number of columns 
defined in the schema definition 161. In other embodiments, templates are not used but the 
"post-parsed SQL" results are dynamically generated from the schema definitions 161 and the 
semantic instance types. 

In this example, the net price corresponds to a fact column in a fact table. This indicates that 
the table entitled SSA in the post-parsed example includes one fact called net price. 

In some embodiments, the pre-parsed templates include additional tokens to deal with 
specific data stores. For example, the “select into” statement is a token in the pre-parsed version. 
This compensates for whether the data store is in Oracle database or an SQL server. 

Another feature of the pre-parsed language is that #begin#" is used to break the pre-parsed 
version into adaptive template blocks. 

Examples 

Appendix A illustrates semantic types that may be supported and their corresponding 
adaptive template names. For example, the Pipelined semantic type is made up of, in this order, 
the map_keys the pipe state and the index fact adaptive templates. The example pre-parsed and 
post parsed SQL adaptive templates are then provided. 
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As mentioned previously, the use of the semantic types significantly reduces the amount of 
work needed to implement the datamart 1 50. By selecting a semantic type for a particular fact 
table or dimension table, the consultant automatically selects the corresponding pre-parsed SQL 
adaptive templates. The selected adaptive templates are then automatically converted into post- 
5 parsed SQL statements that include the schema specific information for the datamart 1 50. 
Additionally, these post-parsed SQL statements include the SQL for converting the data in the 
staging tables 130 into data that can be used in the datamart 150 tables. 

Additional Templates 

Two types of templates not described in Appendix A are “team” templates and 
10 “denormalized” templates. 

The team template is used to properly populate an “associative” dimension table. Such a 
7 table is used whenever there is a one-to-many relationship between an individual fact row and a 

:f| dimension. For example, if multiple salespeople can split the credit for an order, one needs some 

^ way to represent this situation in the datamart. In a star schema, one normally associates a tuple 

15 of dimension values with a fact row (e.g. product, customer, salesrep dimensions for the fact row 

containing price, quantity etc.). Since there is only a single salesrep_key, one could normally 
have only one salesrep associated with this transaction. There are two solutions. One is to 
introduce multiple fact rows for a transaction invovling one to many relationships. If there were 
three salesreps on a specific order, there would be three fact rows for this order stored in the 
20 database. This has the disadvantage of multiplying the data size by a factor of three and slows 
queries. Also queries that are concerned with the total number of transactions become more 
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difficult to process since duplicate rows, due to the multiplication by the number of salesreps, 
must be eliminated. 

Another solution is to introduce an associative table between the actual salesrep dimension 
table and the fact table. Conceptually, the associative table contains “teams” of salespeople. If 
5 salesreps A, B and C often sell products together, they will be associated with a unique team key. 
The team key will be stored in each fact row for orders sold by the A, B, C team. The associative 
table will associate the team key with the three rows for A, B and C in the salesrep table. The 
associative table will have 3 rows representing this team (A-key, team 1 -key), (B-key, team 1 -key) 
% and (C-key, teaml-key). If the team of A, B, D and Q also sold products together, the 
JO associative table would have four additional rows (A-key, team2-key), (B-key, team2-key), (D- 

'"4 key, team2-key), (Q-key, team2-key). The team template scans the staging table used to load 

' J the fact table and generates the appropriate rows for entry into the associative table, only for 
; yfj those teams THAT ACTUALLY OCCUR in the fact rows being loaded. 

W Also, if a team is already present in the associative table it will be reused. 

Hs In real world situations, the number of teams that actually occur is much smaller than the 

total space of all possible teams. 

Note that this team template can be used wherever there is a one to many relationship 
between fact and dimension rows. 

Another example is in a Sales Force Automation system where the fact rows correspond to a 
20 sales “opportunity”. 


Method and Apparatus for Creating a Well- PATENT Inventors: Craig D. Weissman, 

Formed Database System Using a Computer Page 53 Greg V. Walsh and Eliot L. Wegbreit 

Attorney Docket No 20308 702 
ODMA\PCDOCS\SQL I\2280!7\l 



li if- 


An opportunity may be associated with the dimensions of Sales Lead Source, Product and 
Customer Contact. All of these may be one to many relations, amenable to the “team” concept. 

As mentioned above, the other type of template is the denormalized data template. This is a 
variant of the 

5 “Team” template where instead of introducing the extra associative table between the 

dimension and fact tables, the dimension table is a combination of the associative table and the 
actual dimension table. This effectively flattens the data. In the above example the dimension 
table would contain rows like ("Greg Walsh", A-key, teaml-key), 

=■ ("Craig", B-key, teaml-key), ("Ben", C-key" teaml-key), ("Greg Walsh", A-key", team2- 

TO key) etc. Greg Walsh is a member of both teams 1 and 2 and his name (and 
Si other attributes) rather than just his key (A-key) is stored twice. Used judiciously this can 

™ result in faster queries than the associative table case but results in duplicate data being stored. 
The population of both the denormalized team dimension and the associative table are 
difficult to code properly. This is especially true if this is done incrementally (e.g., on nightly 
¥5 extracts) and if you want to be independent of team order (e.g. A, B, C) is the same as (A, C, B). 
Thus, allowing the consultant to simply select this data semantic provides a significant 
improvement over previous systems. 

Runtime Metadata 

Figure 5 illustrates the schema for the runtime environment within the system 100. The 
20 runtime schema 500 represents the schema description for the schema of the running datamart 
150. That is, when the datamart 150 is created or modified, the schema definition is propagated 
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into the runtime schema 500. Thus, the runtime schema 500 allows for the datamart 1 50 to be 
changed without having to rebuild all the tables and repopulate all of those tables. Additionally, 
the runtime metadata 500 provides the support for aggregate navigation. Aggregate navigation 
involves determining which aggregate to use in response to a query. Schema modification and 
aggregate navigation will now be explained in greater detail. 

The schema modification involves comparing the changed schema definition with the present 
schema definition. As will be seen below, an actual table 502 keeps track of all of the dimension 
tables and the fact tables in the datamart 1 50. When a change is made to the schema definition, a 
comparison is made between the old definition and the new definition. The difference between 
these definitions defines the set of tables, columns, and rows that need to be added, deleted or 
modified, in some way. Importantly, the modifications can often be made without losing any 
data in the datamart 150. 

The aggregate navigation process determines which aggregate most closely suits a particular 
query. The runtime metadata 160 keeps track of the aggregates available in the datamart 150. 
The query and reporting program 104 initiates a view of the runtime metadata 500 (in particular, 
the tables holding the aggregate tables definitions). The view results indicate which aggregates 
are available to answer the particular query. The view results are further examined to determine 
the best aggregate to use (the one that most closely corresponds to the query). 

Importantly, the query machinery does not need to be aware of aggregates to be able to take 
advantage of them. Aggregates are simply presented to the query machinery as a solution to a 
query. 
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Additionally, aggregates can use other aggregates to build themselves. This is because the 
schema definition can be used to determine the relationship between aggregates. 

Runtime Metadata List 

The runtime schema 500 includes the following elements: an actual table 502, an actual 
5 column 504, a fact aggregate table 5 12, a fact aggregate dimension 5 14, a dimension base 
aggregate 516, a dimension base aggregate column 518, a datamart letter 510, the dimension 
base 306, the fact table 304, the external table 422, an actual column 504, a physical type 
definition 530, an actual table type 336, an actual column type 540, the physical type 330, a 
□ database physical type 595, the translation string 332, a translation actual 539, a store type 450, a 
MO date 560, a business process 570, an adaptive template profile 580, and a transaction type 590. 

Runtime Metadata Descriptions 

The actual table 502 corresponds to metadata 1 60 that describe which dimension base and 
;4 fact tables “actually” exist in the datamart 1 50. 

The actual table 502 includes the following attributes: an actual table key, an actual table 
15 name, an actual table type, a dimension base key, an external table key, a fact table key, an index 
flag, a mirror flag, and a logical table name. The primary key is the actual table key. The actual 
table name corresponds to the physical name of this table in the database implementing the 
datamart 150. The actual table type is the logical type of this physical table. For example, if this 
is a dimension staging table or a fact staging table. The dimension base key points to the 
20 dimension base table definition that defined the corresponding physical table. The external table 
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key points to the external table definition that defined the physical table. The fact table key 
points to the fact table definition that defined the corresponding physical table. The index flag 
and the mirror flag are used in indexing and mirroring, respectively. The logical table name 
defines the logical name for this table. 

5 The actual column 504 is metadata describing a physical column in a physical table in the 
datamart 150. The actual column table latches this definition information when the physical 
tables are built in the datamart 150. The actual column 504 includes the following attributes: the 
actual column key, an actual column name, an actual column type, an actual table key, a 
S dimension role name, a foreign table key, a group by field, a hierarchy, a list order, a parent 

W hierarchy, a physical type, a primary key, and a time navigation field. The actual column name 

~i: is the name of the physical column in the physical table in the datamart 150. The actual column 

type is the logical type of the column. The actual table key points to the actual table in which the 
actual column lives. The dimension role name is the logical role name of the dimension in the 
= fact table for dimension foreign keys inside of a fact table. The foreign table key points to the 

ft actual dimension base tables in the actual tables 502 (the foreign table key is applicable to fact 

actual columns that are foreign keys to dimensions). The group by field, for dimension table, is 
true when this column should be included in an aggregate builder group. The hierarchy for 
dimension, for dimension columns, indicates that aggregate builder group to which this column 
belongs. The list order is the order of the column in the actual table 502. The parent hierarchy, 
20 for dimension columns, indicates the parent aggregate builder group to which this column 

belongs. The physical type is a logical data type of the column. The primary key, for dimension 
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tables, is true when this column is the primary key of the actual table 502. The time navigation 
field, for the database dimension, is true if this field can be used by the time navigator. 

The fact aggregate table 512 includes a list of fact aggregates in the datamart 150. The fact 
aggregates includes attributes that point to the actual fact table in which this aggregate belongs. 

5 The fact aggregate table 512 indicates which numbered aggregate represents the fact table in 
question, the number of rows in this aggregate, a datamart letter, and an enabled flag. The 
datamart letter indicates the mirrored datamart to which this fact aggregate belongs. 

Mirror is used to ensure that partially completed extractions from the source systems 1 10 do 
^ not cause the database to become inconsistent. 

10 The fact aggregate dimension 514 lists which aggregates contain which dimensions. The fact 
aggregate dimension includes the following attributes: an actual dimension role key, a dimension 
■ base aggregate key, a fact aggregate dimension key, and the fact aggregate key. The actual 

m dimension role key is the dimension foreign key in this fact aggregate that is being defined. The 

fff dimension base aggregate key is the dimension aggregate that this fact aggregate points to for 

ft this foreign key. The fact aggregate dimension key is the primary key. The fact aggregate key 

points to the fact aggregate being defined. 

The dimension base aggregate table 516 lists all the dimension aggregates in the datamart. 
The dimension base aggregate includes the following attributes: an actual table key, an aggregate 
number, an aggregate size, a datamart letter, a dimension base aggregate key, and an enable flag. 
20 The actual table key points to the physical header for this dimension base. The aggregate 
number, for the dimension table in question, is the number of this particular aggregate. The 
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aggregate size is the number of rows in the aggregate. The datamart letter indicates which 
mirrored database this aggregate lives in. The dimension base aggregate key is the primary key. 
The enable flag indicates whether or not the aggregate navigator should work with this aggregate. 

The dimension base aggregate column 518 is a list of columns in a given dimension 
5 aggregate. The dimension base aggregate column includes attributes which point to which 
column is included in this dimension aggregate, and a pointer to a dimension aggregate being 
defined. 

The datamart letter 510 indicates which of two mirrored datamarts a particular aggregate 
% belongs to. This is an optional element which may not be required if mirroring does not occur in 
id the datamart 1 50. Mirroring duplicates the tables in the datamart 1 50. Changes can then be 
s ! made to one copy of the datamart 1 50, while the other datamart 150 continues running. These 
changes can then be propagated when possible. 

\fi The actual column type 540 is a logical description of role a column plays in the system 100. 
=n The actual column type 540 includes attributes that define the default value to be used in a 
fS database for a column of this type. 

The physical type definition 530 defines which physical types are allowed for which table 
types. The physical type definition 530 includes attributes which point to an actual table type. 
The actual table type is a logical type of a physical table (for example, dimension, fact etc.) being 
defined. The physical type definition also includes an attribute that indicates whether to select 
20 this item by default when giving the consultant or user a choice. 

The database physical type 595 defines the name of the physical database. 
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The translation actual table 539 defines the actual values of translations strings for a single 
relational database management system. These translations strings are the real strings to use for 
a given translation string within a store type. The translation actual table 539 also includes 

attributes that point to the store type. 

Figure 5 also illustrates additional tables used in the system 100. 

The date table 560 is used to track date information in the datamart 150. Importantly, times 
and dates are always treated corrected in the datamart 150. This can be guaranteed because the 
consultant cannot change the definition of dates in the datamart 1 50. Thus, for example, the 
month of September will always have 30 days, and leap years will be handled correctly. 

The transaction type table 590 is a list of the available transaction types within the system 

100 . 

The adaptive template profile 580 is used as a communications mechanism for templates. 

The adaptive template profile 580 includes a number of rows being communicated back to the 
calling program. The adaptive template profile 580 also indicates the logical name for 
information being communicated back from an adaptive template to the calling program. 

The business process table 570 is a look up table for supported business process types during 
the extraction. The business process 570 includes a business process key and a process name. 
The process name corresponds to a logical name for a business process to which fact staging 
table belongs. The process key identifies a business process record in a fact staging table. 
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Time Navigation 


An important feature of some embodiments of the invention is the ability to compactly store 
and rapidly query “historical” backlog/balance/inventory quantities. By historical we mean the 
amount of backlog or inventory as it existed at a given point in time-not necessanly today. Note 
that backlog/balance/inventory quantities are different than transactional quantities. For 
example, your bank balance at the end of Q1 1997 is not the sum of your bank balances at the 
end of Jan, Feb and March. It is computed by adding all of the deposit transactions and 
subtracting all the withdrawals from the balance at the end of Q4 1996. One could compute 
balances by this method when a user queries the system but because this method requires rolling 
forward all of the appropriate transactions “from the beginning of time, the queries will likely 

be slow. 

The traditional solution in datamarts is to store periodic “snapshots” of the balance. The 
snapshots are often stored at daily intervals for the recent historical past, and at greater intervals 
(e.g. weekly or monthly) for less recent history. This approach has two big disadvantages. The 
first is an enormous multiplication of data volume. If, for example, you are keeping track of 
inventory in a store you must store a snapshot for each product you hold m inventory for each 
day, even if you only sell a small fraction of all of your products on a given day. If you sell 
1 0,000 different products but you only have 500 transactions a day , the “snapshot datamart is 20 
times larger than the transactional datamart. The second disadvantage relates to the most 
common solution for alleviating the first problem, namely storing snapshots at less frequent 
intervals for less recent history. This results in the inability to compare levels of inventory in 
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corresponding time periods since the same level of detail is not present in earlier data. For 
example, in manufacturing companies it is often the case that much business is done near the end 
of fiscal quarters. If one wants to compare inventory levels between Q1 1995, Q1 1996 and Q1 
1997, and focus on the most important changes which occur near quarter end, one cannot use the 
5 approach of storing the snapshots at coarser levels of detail since daily data would be required. 

In some embodiments of the system, the aggregate tables are used to answer queries about 
backlog/balance/inventory quantities. In order to answer such queries, the previously described 
rolling forward from the beginning of time is done. However, this is performed efficiently 
S through the accessing of the appropriate time aggregates. For example, assume the datamart 150 
td has five years of historical transaction data beginning in 1 993 . Assume that one desires the 

inventory of some specified products on May 10, 1996. This would be computed by querying all 
of the transactions in the 1 993 , 1 994 and 1 995 year aggregates, the 1 996 Q 1 quarter aggregate, 

[S the April 1996 month aggregate, the May 1996 week 1 aggregate and finally 3 days of actual 
iy) May, 1996 daily transactions. These transactions (additions and subtractions from inventory) 
lb would be added to the known starting inventory in order to produce the inventory on May 1 0. 
Note that this “time navigation “hops” by successively smaller time intervals (year, quarter, 
month, week, day) in order to minimize the number of database accesses. What is important is 
the exploitation of aggregate tables, that already exist in the system in order to answer 
transactional queries rapidly (e.g. What were the total sales of product X in Apr 1996?). This 
20 avoids the need to build what is essentially a second data datamart with the 
balance/inventory/backlog snapshots. 
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Query Mechanism Metadata 

The following describes the metadata 160 used in the query/reporting program 104. This 
metadata is shown in Fig. 6. Generally, the query mechanism metadata can be broken into 
ticksheet metadata, measurement metadata, filtering metadata and display options metadata. The 
5 ticksheet metadata defines the user interface objects for user interaction with the datamart 1 50. 
The ticksheet defines how users can initiate queries and how results are presented back to the 
user. The measurement metadata defines a logical business calculation that can be presented to a 
user. Typically, the measurement metadata defines a format for presenting information to user 
V that is more easily understood by the user or provides a more valuable result to the user. The 

id filtering metadata defines how a user can filter results. Filtering allows the results set to be 

2 limited to particular dimension values. The display options metadata defines display options that 
can be provided to the user. 

H The following describes some important features of the user interface. The user interface 

| allows the user to drill down through data. Also, portions of the query forms can be dynamic 

ft based upon values in fields (e.g., a list box can be dynamically updated because it is tied to a 

field in the datamart 150, that when changed, cause the values in the list box to change). Also, a 
query is guaranteed to be consistent with the schema because the query is tied to the schema 

definition. 
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uerv Mechanism Schema List. 



Figure 6 includes the following elements: the constellation 302, a ticksheet 602, a data set 
606, a ticksheet column 608, a tip 601, an attribute role 603, an attribute 610, a ticksheet attribute 
605, a ticksheet type 604, a measure 620, a measure term 630, a measure unit 624, a term 


5 operator 632, a transaction type 590, an RPN operator 636, the fact column 310, the fact table 
304, a backlog 638, a measure mapping 622, a ticksheet column element 612, a dictionary 640, a 
filter block 650, a filter block type 652, a filter group 654, a filter element 656, a ticksheet type 
option 660, an option location 662, an option value 664, an option name 666, an option display 
y type 668, an application type 691, the dimension role 320, the dimension column 329, and the 
IS dimension base 306. 

y Query Mechanism Schema Metadata Descriptions 
/! Ticksheets Metadata 

y Under the constellation 302, the ticksheet is a top level object for defining the user interface 

Pi objects for user interaction. The ticksheet 602 table includes a data set key, a name, a ticksheet 

15 type, a constellation key, a label, label detail, a list order, a cleanse flag, and a description. The 

cleanse flag indicates whether or not to cleanse the filter data on this ticksheet. The constellation 
flag indicates the constellation in which the dimensions and measures for this ticksheet reside. 
The data set key indicates the page in which the end user makes report selections. The data set 
key represents a logical grouping of similar ticksheets. The description is for documentation 
20 purposes. The label is the string for the name of the ticksheet. The list order indicates the 
ticksheet list order. The name is the hidden name of the ticksheet. The ticksheet includes a 
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primary key. The label detail allows for more verbose documentation. The ticksheet type 
indicates the type of application that interprets the selections made on this ticksheet. 

The data set table 606 is a grouping mechanism for ticksheets into sets that describe their 
contents. The data set table 606 includes the following columns: a data set key, a data set name, 
a label, a description, and a list order. The data set name is a logical name for top level user 
definition of like ticksheets across ticksheet types. The description is the description of the data 

set. 

The ticksheet column 608 defines a single column for displaying measure choices on a 
ticksheet. The ticksheet column table 608 includes the ticksheet column key, the list order, the 
ticksheet key, and the description. These columns and the ticksheet column table 608 operate in 
a manner similar to such columns in other tables in this metadata. 

The ticksheet column element 612 is a single value within a measure display column on a 
ticksheet. The ticksheet column element 612 includes an abbreviation, a description, a dictionary 
key, a label, a list order, a name, a ticksheet column element key, and a ticksheet column key. 

The ticksheet column element key is the primary key for this table. The ticksheet column key 
points to the ticksheet column 608 entry. The name is the hidden name for the column element. 
The abbreviation is the shortened name for the user display. The dictionary key is the key for 
help text entry for this element. It is a key into the dictionary 640. The other columns act in a 
manner similar to columns in other tables of similar names. 

The tip table 601 includes a ticksheet key, a name, a description, and a list order. The tip is a 
definition of user tips for using a particular ticksheet. 
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The attribute table 610 defines the dimension attribute choices within a ticksheet. These 
choices are tied to a single dimension column in the schema definition of the datamart 150. The 
attribute table 610 includes an abbreviation, an attribute key, a dictionary key, a dimension 
column key, a dimension role key, a hyperlink, a label, a list order, a name, and a ticksheet key. 
The abbreviation is the shortened user string for the attribute. The attribute key is the primary 
key for this attribute. The dictionary key is a pointer to the dictionary 640 that includes help 
message for a particular attribute. The dimension column key is the dimension column in which 
this attribute refers. For degenerate dimensions this reference is null. The dimension usage, 
within a constellation, is defined by the dimension role key. The hyperlink is an html text for 
navigating return values for this attribute to other web sites, such as a company name look-ups 
etc. The label is what the user sees for a particular attribute. The list order defines a sort order 
on pop-up menus where one is the topmost in the list. The name is the internal name for the 
attribute. The ticksheet key indicates the ticksheet to which this attribute belongs. 

The attribute role 603 table is a lookup table list of valid roles for attributes within a ticksheet 
type. The attribute role includes the attribute role key and the ticksheet type. 

The ticksheet attribute 605 indicates the roles played by dimension attributes within a 
ticksheet. The ticksheet attribute 605 includes the attribute key, an attribute role, a ticksheet 
attribute key, and a ticksheet key. The attribute key indicates the attribute in the attribute table 
610 which has a role define on the ticksheet. The attribute role is the role being granted. The 
ticksheet attribute key is the primary key. The ticksheet key is the ticksheet being defined. 

The application type 691 is the top level grouping for ticksheet types. 
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The ticksheet type table 604 lists the applicable applications that can use ticksheets. 

Examples may be simple reporting applications or relevancy applications or trend type of 
applications. The ticksheet type 604 includes the application type, a ticksheet type, a template, 
and a ticksheet type name. The application type is the definition of the high level application in 
5 which a particular ticksheet type resides. The ticksheet type is the logical name for a program 
that interprets ticksheets. The template is the template for the ticksheet. The ticksheet type name 
is the name displayed for a particular ticksheet type. 

Measurement Metadata 

% The following describes the measures used in the query mechanism schema 600. The 
S measure table 620 defines a top level object for a logical business calculation. The measure table 

:j 620 includes a constellation key, a description, a measure key, a measure unit, and a name. The 

M constellation key points to the constellation in which the measure resides. The description is for 

y ; j documentation purposes. The measure key is the primary key for the measure table 620. The 

m measure unit is an indicator of the manner in which numbers are to be displayed. The name is 
the logical name of the measure. 

The measure unit table 624 is a lookup table of the valid list of measure unit types. An 
example of such unit types is currency. 

The measure term table 630 indicates a single component of a measure. The measure term 
can be combined arithmetically to construct a measure. The measure term table 630 includes a 
20 backlog type, a fact column key, a fact table key, a list order, a measure key, a measure term key, 
an RPN operator, a term operator, and a transaction type key. The backlog type indicates the 
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type of backlog operation to use for a particular term (e.g., “beginning of period” and “end of 
period”). This can possibly be none. The backlog types are defined in the backlog type table 
638. The fact column key points to the particular numeric column to operate on in the fact table. 
The fact table key indicates the fact table being operated on. The list order is the order of this 
5 term in the measure 620. The measure key is the measure being defined. The measure term key 
is the primary key for this table. The RPN operator is for the measure terms that perform 
arithmetic operations on other terms. (The RPN operator table lists the valid arithmetic 
operations to use when constructing a measure.) The term operator is an SQL operator to use on 
if a set of fact rows. (The term operator table 632 indicates the valid set of SQL operators to use on 

: 1 10 a measure term.) The transaction type is the transaction type values to filter on for the fact in 

question. 

The relation between measures and ticksheets is handled through the measure mapping table 
i; 622. The measure mapping table 622 includes the measure key, the ticksheet key, and the 
fi combination ID. The measure key points to the particular measure that is related to the particular 
15 ticksheet. The combination ID identifies a set of ticksheet column elements being defined. 




Filtering Metadata 

Filtering allows results to be limited to only particular dimension values. For example, a user 
may want to limit the results to particular customer names. 

The following describes the filtering tables. The filter block 650 is a top level grouping table 
20 for filter area within a ticksheet. The filter block 650 is tied to a particular dimension column in 
the schema definition. The filter block 650 includes, columns, a description, a dictionary key, a 
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dimension column key, a dimension role key, a filter block key, a filter block type, a label, a list 
order, a name, a plural, a mapping flag, and a ticksheet key. The columns field indicates the 
number of columns in this filter block. The description is for documentation. The dictionary key 
points to the help dictionary. The dimension column key points to the actual column name and 
5 the datamart to be filtered on. A null value here means degenerate dimension as determined by 
the dimension role key. The dimension role key points to the dimension role in the constellation 
of the ticksheet that is the form key to filter on for all facts in this constellation. A null value 
here means that a special dimension shared by all constellations is being used. The filter block 
key is the primary key for this table. The filter block type points to the filter block type table 652 
10 which defines the ways in which this filter block is displayed to the user (e.g., a checkbox or a 
radio button). The label is the text that appears to the user for the filter block. The list order is 
the order that the filter block should appear in a list. The name is the name of the filter block. 

The plural field is the text that appears to the user for the filter block. The mapping flag is used in 
mapping. The ticksheet key points to the ticksheet that this filter block belongs. 

15 The filter group 654 is a mid-level grouping for a filter on a ticksheet. This groups individual 
selections into logical units. 

The filter element table 656 defines individual values for a dimension attribute within a filter 
block. The filter element table 656 includes a dictionary key, a filter element key, a filter group 
key, a label, a list order, a name, an SQL statement, and a value. The dictionary key points to the 
20 user help text for a particular filter element. The filter element key is the primary key. The filter 
group key points to the filter group to which this element belongs. The label is the user 
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displayed string for the element. A list order is the order of this element within a filter group. 
The name is the hidden name of this element. The SQL statement is an SQL statement used to 
build the list of values for a dynamic list box filter group. The value is the database value that 
this element translates into in a SQL "WHERE" clause. 

Display Options Metadata 

To control the method in which information is displayed with a ticksheet, a set of options are 
supplied. The ticksheet type option table 660 helps support this feature. The ticksheet type 
option table 660 includes a list order, an option display type, an option location, an option name 
key, a ticksheet type, and a ticksheet type option key. The list order is the order to display 
options for a ticksheet type. The option display type is an html control type to use when 
displaying a particular option. The option location is the location of the option on the ticksheet. 
(The option location table 662 holds the list of possible locations of options on a ticksheet.) The 
option name key is the option being included in a ticksheet type. (The option name table 666 
defines an option that has meaning for one or more applications that can be used by users.) The 
ticksheet type is the ticksheet type being defined. The ticksheet type option key is the primary 
key for this table. 

The option name table 666 includes option name key, a name, a label, and a dictionary key. 
The option name key is the primary key. The name is the logical name for the option. The label 
is the label seen by the user as the name of the option. The dictionary key is the pointer to the 
help text dictionary. 
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The option value table 664 defines single valid values for options. The option value table 
664 includes the dictionary key, a label, a list order, an option name key, an option value key, 
and a value. The dictionary key is the help text dictionary key. The label is the label seen by the 
user for this option choice. The list order is the order of the valid values for the option. The 
option name key is the option set being defined. The option value key is the primary key for this 
table. The value is the hidden value for the option. 

The option display type table 668 is a lookup table indicating the valid way that options can 
be displayed. 

The dictionary table 640 is a table for help text for users. 

User Interface Example of Defining Metadata 

General Schema Definitions User Interface 

The following describes a constellation used in a business. In this example a new dimension 
is added very simply and the changes are automatically propagated into the datamart 150. The 
enterprise manager interface 192 is used by the consultant to define and manipulate the system 
100 . 

Figure 7 illustrates the enterprise manager interface 192. Multiple system 100's can be 
connected to through that interface. Many of the objects and tables in the system 100 are shown. 
The base dimensions definitions 710 correspond to the base dimensions available under the 
“epitest” datamart. The constellations 712 for this datamart include an expense constellation and 
a sales constellation 720. Thus, the sales constellation 720 would appear as a row in the 
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constellation table 302. Under the sales constellation 720 appear the definitions for the sales 
aggregates 721, the sales dimensions 723, the sales degenerate dimensions 725, the sales facts 
726, the sales measures 728, and the sales ticksheets 729. Also, the extraction definitions 740 
and security definitions for the “epitest” datamart are accessible. The sales dimensions 723 
5 define rows in the dimension role table 320. These rows include customer billed to, product, 
application, program, customer ship to, and territory. 

Figure 8 illustrates the dimension table definition window 800 (presented to the consultant as 
the result of selecting the customer billed to dimension role under dimensions). A dimension 
S table definition window 800 show that the dimension 820 is customer bill to and the base 

To dimension 8 1 0, to which the dimension 820 points to, is named customer. 

£ Figure 9 illustrates a base dimension window 900 showing the definition of the base 

dimension 8 1 0 named customer. In this case, the customer base dimension has a “slowly 
iJl changing dimensions” dimension data semantic 910. In this example, the dimension base 810 

£ customer has a number of dimension columns 920. LI is an example of a dimension node. 

'15 Figure 10 illustrates the dimension column window 1000 for the customer region code 
column 1010. The physical type is the type of data defining that dimension column. The 
VARCHAR_50 physical type is then mapped to an actual type through the physical type table 
330. The translation is dependent on the store type. 

Figure 1 1 illustrates the base dimension window for the base dimension date (substantially 
20 un-editable). The user interface indicates that the date dimension is not an editable base 
dimension (shown as black circles under “Base Dimensions”), and grayed out in the base 
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dimension window 900. The transtype (transaction type) is similarly not editable and is similarly 
shown in the user interface. 

Figure 12 shows the dimension column “date day quarter end”. Note that column cannot be 
edited. 

5 Figure 1 3 illustrates a fact table window 1 300 that is open on the order fact table 1310 
definition. The fact data semantic 1 3 1 0 is transactional/state like/force close/unjoined. The 
transactional/state like/force close/unjoined means that the invoice part of an order is 
transactional, the booking is state like, orders that are not otherwise dealt with, are closed out, 

% and the data may become dirty and so it needs to be cleansed, thus, it is unjoined. This semantic 
iio type is described in detail in Appendix A. Note that the user can select from many different types 

^ of fact table semantics. The fact table window 1 300 also shows the fact columns 1 330 for the 

order fact table. 

\r\ Figure 14 illustrates a fact column window 1400 opened on the definition of the netjprice 

S’! fact column 1410. Here the fact column 1410 has a physical type 1420 called FACTMONEY and 
T5 an aggregate operation 1430 called a SUM. 

Figure 15 illustrates how the consultant would select the semantic type for the order fact table 

1310. 

Figure 16 illustrates the results of a request by the consultant to generate the datamart 1 50 
from the definitions of the datamart. The results show that a number of tables have been created 
20 in the datamart 150. Importantly, Figure 16 illustrates the results of an initial build process. In 
subsequent modifications, only those elements of the datamart that have changed will be 
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changed. In other words, the subsequent changes are handled as an update process. An example 
of the update process is described below. 

Extraction Interface Elements 

The following describes the creation of the connectors 162. Once the schema definitions 161 
5 are set, the consultant then defines the connectors 162 to the source systems 110. The 

connectors, as noted above, define how information is to be extracted from the source systems 
1 10 and how that information is to be placed into the datamart 150. 

These connectors are defined under the extraction definitions 740. Figure 17 illustrates the 
|S; job definition window 1 700 presented when the consultant has selected a particular job. 
l (jj Figure 1 8 illustrates the job steps 1810 within the default job. The checkbox indicates 

fi! whether the particular job step is enabled for that job. The list of job steps is shown in the order 
^ that they are executed. The two foreign keys within the job step are shown in the dialog box of 

jij Figure 17 to indicate whether the job step is a connector or a system call. 

3 Figure 1 9 illustrates the All Semantics connector as defined in the connector definition 

15 window 1900. This connector includes the description and a definition of the input and output 
data stores. In this case, both of the data stores are the “epimart” (which is the datamart 150). 

Figure 20 illustrates the data store window 2000 interface for showing a data store. This is 
the data store that is referenced in the connector All Semantics. 

Returning to the discussion of connectors. Figure 21 illustrates the connector entitled MFG. 
20 The MFG connector has two major steps: (1) order dimension staging, and (2) order fact staging. 
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The results of these extraction steps are put in the staging tables 130. (The all extraction steps 
window 2100 illustrates all the possible steps in the system that can be used.) 

Figure 22 illustrates the SQL statement window 2200. The SQL statement window 2200 has 
an SQL field 2210 that includes the SQL statements that loads a customer table. As shown in the 
5 dialog box, the table references for the SQL statement includes the customer dimension table 
column definitions. That is, this SQL statement is going to be used to populate the customer 
dimension table. 

In this example, the base name, type code, type name, region code, region name and tier 
% name corresponds to the column names within the customer dimension. The date modify is an 
l'S additional field that is to be used to indicate when this field was last modified in the database. 
Additionally, there is a source system key that is automatically included in every dimension. The 
^ source system key helps ensure that the datamart 1 50 is well-formed. 

■ * In one embodiment of the invention, these names can be automatically propagated into the 
Tj SQL field 2110 window via a template that is generated from the corresponding base dimension. 
lSl This allows the consultant to more easily define the SQL selection statement. 

Figure 23 illustrates the SQL statement for the Open Order Stage for populating the order 
fact table. 

At this point the steps for generating the staging table information are complete. Now the 
semantic conversion steps are defined. 
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In Figure 24, returning to the connector steps window, we have switched to an All Semantics 
connector 2410. The All Semantics connector 2410 causes the semantic conversion of the 
information in the staging table for use in the datamart 150. 

Figure 25 illustrates the semantic transformation window 2500 showing the dimension table 
5 customer semantic 2510. 

Figure 26 illustrates the order fact semantic 2610 definition. 

Figure 27 illustrates the results of a consultant adding a new dimension 2700 (called 
warehouse) to the sales constellation 720. The batch operation window 1600 illustrates the 
changes that are being made to the datamart that was created in Figure 16. To achieve these 
M results, the consultant need only perform the following steps: 

%.j 1 . Define the new dimension. 

: y 2. Define the connector steps, including the SQL Statement to extract the warehouse data 

:=! from the source systems 1 10. 

Ifj 3. Add the warehouse information to the Open Order Stage SQL Statement. 

15; 4. Define a semantic transformation for the warehouse, e.g., slowing changing dimension. 

5. Have the enterprise manager 102 update the datamart 150. 

Thus, changing the schema definition of the datamart 150 is significantly simpler than 
previous systems. 

Additional Interface 

20 Figure 28 illustrates the aggregate group window 2800, where aggregates can be defined. 

For a given aggregate group, the consultant can define which fact share the aggregate, and which 
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type of aggregate should be built for a given dimension in the aggregate. Additionally, 
dimensions can be added to, or removed from, an aggregate group. 

Figure 29 illustrates a portion of the configuration window 2900. In this example, a partial 
list of the transaction types 2910 is shown. Thus, the consultant can determine which transaction 
5 types will be available to him/her. 

End User Interface Definition and Example 

Figure 27 illustrates the interface used to define a user interface for the end user. Figure 30 
includes a user interface definition window 3000 which can be used to define measures and 
0 ticksheets. In this case, the measure definition window 3010 is shown. 

IQ The measure definition window 3010 allows the consultant to define which measures will be 
;£> available in the system. The consultant defines the name, units, and constellation for a particular 
: =j measure. The measure is further defined by defining the list of measure terms that make up a 
□ measure (the calculations for the measure 3020). In this example, the ASPBacklogOrderGross 
d measure has seven calculation steps, some of them arithmetic (e.g., SUM) and others RPN 
15 (Reverse Polish Notation). 

Figure 3 1 illustrates the ticksheet definition window 3 1 00. The ticksheet definition window 
allows a consultant to define a ticksheet that will be used to generate a query form for a user. 

The consultant defines the attributes, the columns, and the filters for a ticksheet. Figure 32 
illustrates the query form 3200 generated from the ticksheet defined in Figure 3 1 . 

20 Figure 33 illustrates the measure mappings window 3300, that allows the consultant to map 
measure definitions to user friendly measure names. In the example of Figure 33, the 
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PriceShipGrossMonth measure is mapped to a combination of the dollar amount, gross, and sell- 
through being selected in the query form 3200. 

Figure 34 illustrates another query form 3200 generated from a different ticksheet definition. 
When the user selects the create report button, the query is issued against the datamart 150. 

5 Figure 35 illustrates some sample results from such a query. 

The following query log illustrates the actual query that was executed against the datamart 
1 50. The query log illustrates that an aggregate and navigation process determined which 
aggregate would be the most appropriate. The aggregate builder had created these aggregates. 
The most appropriate aggregate for the requested query was selected. The results were then 
ii): returned. 


Query log 


time : <A Date Here> 

addr : 192.0.0.210 

host : 192.0.0.210 

user : 

agent : Mozilla/4 . 01 [en] (WinNT; U) 


Datebase information: DRIVER={SQL SERVER) ;SERVER=bigfoot;DATABASE=macromedia 


Keys and values coming in from the browser: 
file = 
fileDesc = 
queryaction = QUERY 
hidden_queryaction = QUERY 
OK_callback = 

NOK__callback = 
ticksheet = Orders 
hidden_ticksheet = Orders 
Rows = Customer 
hidden_Rows = Customer 
Columns = Fiscal Year ' 
hidden_Columns = Fiscal Year 
units = Price 
hidden_units = Price 
facttype = Shipped 
hidden_facttype = Shipped 
facttype2 = Gross 
hidden_facttype2 = Gross 
stage = Orders 
hidden_stage = Orders 

currencyunits = Thousands 
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hidden_currencyumts = Thousands 
rowtotal = yes 
hidden_rowtotal = yes 
columntotal = yes 
hidden_coluiantotal = yes 
percent = none 
hidden_percent = none 
precision = 0 
hidden_precision = 0 
charts = 3D 
hidden_charts = 3D 
maxrows =10 
hidden_maxrows = 10 
rowsorttype = value 
hidden_rowsorttype = value 
Fiscal_Years = All 
hidden_Fiscal_Years = All 
Fiscal_Quarters = All 
hidden_Fiscal_Quarters = All 
Calendar_Months = All 
hidden_Calendar_Months = All 
Business_Units = All 
hidden_Business_Units = All 
Product_Lines = All 
hidden_Product_Lines = All 
Product_Supergroups = All 
hidden_Product_Supergroups = All 
Platforms = All 
hidden_Plat forms = All 
Product_Languages = All 
hidden_Product_Languages = All 
Product_SKUs = All 
hidden_Product_SKUs = All 
Product_SKU = 

Sales_Reps = All 
hidden_Sales_Reps = All 
Channels = All 
hidden_Channels = All 
Customer_Types = All 
hidden_Customer_Types = All 
Customer_Regions = All 
hidden__Customer_Regions = All 
Customers = All 
hidden__Customers = All 
Customer = 
sqStyle = classic 
hidden_sqStyle = classic 

Contents of %FormData: 

Columns = Fiscal Year 
rowsorttype = value 
ticksheet = Orders 
Customers = All 
charts = 3D 
Customer Types = All 
Product SKUs = All 
Customer Regions = All 
Fiscal Quarters = All 
Rows = Customer 
Channels = All 
precision = 0 
Product Supergroups = All 
percent = none 
maxrows = 10 
Sales Reps = All 
columntotal = yes 
Calendar Months = All 
queryaction = QUERY 
sqStyle = classic 
Fiscal Years = All 

Product Languages = All 
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Platforms = All 
Product Lines = All 
rowtotal = yes 
currencyunits = Thousands 
Business Units = All 

The colheaders are: 


The Cellitems are: 

Price Shipped Gross Orders 

The Cellitems abreviated are: 

Dollar Amount Shipped Gross Orders 

pid is: 310 
spid is: 25 

The valid collheaders are: 

The invalid colheaders are: 

The valid cellitems are: 

Price Shipped Gross Orders 
The invalid cellitems are: 

The unitstack is: 

CURRENCY 

The cellstack is: 

-SUM (Order . net_price ) 

The selectstack is: 

-SUM ( Order . net_price ) 

The typestack is: 

SHIP 

Transtypes are: 

"BEGIN_RETURN" = 1007 
"END_GROSS" = 1004 
"END_SRBOTH" = 1018 
"END_SRADJ" = 1012 
"BOOK" = 1 
"BEGIN_ANET" = 1013 
"END_IADJ" = 1024 
"END_ICOMP" = 1022 
"BEGIN_GROSS" = 1003 
"BEGIN_SRBOTH" = 1017 
"END_SBOTH" = 1016 
"END" =1002 
"BEGIN_SRADJ" = 1011 
"END_SADJ" = 1010 
”BEGIN_IADJ" = 1023 
"BEGIN_ICOMP" = 1021 
"END_NET " = 1006 
"SHI P_AD JUST" = 103 
"LOST" = 3 
"END_SALL" = 1020 
"BEGIN_SBOTH" = 1015 
"BEGIN" = 1001 
"BEGIN_SADJ” = 1009 
"BOOK_RETURN" = 2 
"END_FADJ" = 1026 
"BEGIN_NET" = 1005 
"BEGIN_SALL" =1019 
"GL" = 105 
"SHIP_RETURN" = 102 
"SHIP_RADJ" = 104 
"SHIP" = 101 
"END_RETURN" = 1008 
"INV_ADJUST" = 201 
"LEAD_LOST" = 4 
"BEGIN_FADJ" = 1025 
" FINV ADJUST" = 202 
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"END_ANET" = 1014 ' 

The facttable is: 

Order 

The limitvalues are: 

The access limitations are: 

The limitvalues are: 

The header took 261 milliseconds. 

Parsing took 160 milliseconds. 

Generating the header took 0 milliseconds. 

Building user limits took 0 milliseconds. 

Phase 0: Aggregate navigator initialization. 

Phase 1: Getting dimensions from SQL. ( 10 ) 

Phase 2: Getting fields available from SQL. ( 90 ) 

Phase 3: Getting degenerate fields from SQL. ( 0 ) 

Phase 0: Preparing for query building and execution. R2 [0 -> 0] = (SUM(ZO)) 
Rl [ 0] = SUM (Z0) 

The column router: 

Cell location 0 will be returned in column 0 when Type is SHIP. 

The result router: 

Result location 0 is (SUM(ZO)) 0 

The unique facttables are: Order 

The number of unique facttables are: 1 

The unique types are: SHIP 
Table to unique number lookup 
Order => _0_ 

Begin work on the query based on the facttable Order 
Phase 1: Table Order. Creating table aliases ( 10 } 

JOIN_FIELD IS CUSTOMER_BILLTO_KEY FOR Customer 
JOIN_FIELD IS FOR Fiscal Year 

SQL table aliases : 

OrderL : T3 
DateC : T2 

CustomerDCUSTOMER_BILLTO_KEY : T1 
Table aliases: 

tablealiaslookup (Tl) = Customer 
tablealiaslookup (T2) = Date 
tablealiaslookup (T3) = Order 
selectalias : 

Customer: Tl.base_name 
Fiscal Year: T2.fy_name 
selectstackalias : 

-SUM (Order. net_price) : -SUM (T3 . net_price ) 
joinalias : 

Customer: Tl . customer_key = T3 . customer_billto key 
Phase 2: Building SELECT clause ( 0 ) 

Phase 3: Building FROM clause ( 0 ) 

Phase 4: Building WHERE clause ( 0 ) 

Phase 5: Building GROUP BY clause ( 0 ) 

SQL before going through the aggregate navigator: 

SELECT 

Columns = T2.fy_name, 

Type = T3.Transtype_key, 

CO = -SUM(T3.net_pnce) , 

Rows = Tl.base_name 
INTO #tmp_0_ 

FROM 

Customer Tl, 

Date T2, 

Order T3 
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WHERE 

Tl.customer_key = T3 . customer_billto_key and 
T2.date_key = T3.date_key and 
T3.Transtype_key in (101) 

GROUP BY 

T1 . base_name, 

T2 . fy_name, 

T3 . Transtype_key 


************************************************************^*^*^^ + ^^^ 
Selecting appropriate aggregate for the query. 

Phase 0: Aggregate navigator. Preparing for query building and execution. 
Phase 1: Spliting query into clauses. ( 0 ) 

Phase 2: Construction of aliases. ( 0 ) 

Phase 3: Extracting neededfields from where clause. ( 0 ) 

Phase 4: Extracting neededfields from group by clause. ( 0 ) 

Phase 5: Extracting neededfields from select clause. ( 0 ) 

Phase 6: Unaliasing. ( 0 ) 

Phase 7: Constructing the SQL to fetch smallest aggregate. ( 20 ) 

Phase 8: Running the big SQL. { 20 ) 

Phase 9: Extracting results from the big SQL. ( 0 ) 

Phase 10: Adjusting input with aggregate information. ( 0 ) 

Phase 6: Aggregate Navigating ( 40 ) 

********************* ******************************************************** 
Appropriate aggregate determined (CUSTOMER_0, DATE_4 , ORDER_86) , now select 
*******************************************^^ A . + ^ if ^. + A ,^^. :fc ^^ + ^^^^^^^ A ^ + ^^^^^^^^^ 

SQL after going through the aggregate navigator: 

SELECT 

Columns = T2.fy_name, 

Type = T3 .Transtype_key, 

CO = -SUM (T3 .net_price) , 

Rows = Tl.base_name 
INTO # tmp_0_ 

FROM 

CUSTOMER_0 Tl, 

DATE_4 T2, 

Order_86 T3 
WHERE 

Tl.customer_key = T3 . customer_billto_key and 
T2.date_key = T3.date_key and 
T3 . Transtype_key in (101) 

GROUP BY 
Tl .base_name, 

T2 . fy_name, 

T3 . Transtype_key 


Phase 7: Building results table in sql ( 10435 ) 
Phase 15: Splitting tables by type {needed =0) {0 

Phase 16: Merging results into one table (needed = 0) 


( 0 )GR_COLS = CO = SUM (CO) 


SQL: SELECT Rows INTO #tmpAllRows FROM #tmpjD_ GROUP BY Rows ORDER BY SUM (CO) DESC 
SQL: SELECT count (Rows) FROM #tmpAHRows 

Phase 17: Extracting rows and doing number of total records (10508/10498) ( 1022 ) 

SQL: set rowcount 10 

SELECT Rows INTO #tmpTopRows FROM #tmpAllRows 
set rowcount 0 

Phase 19: Sorting, Top (needed = 10498) { 10 ) 

— creating row totals 
SELECT 

#tmp_0_. Rows, 

CO = SUM (CO) 

INTO #tmpRows 

FROM #tmp_0_, #tmpTopRows 

WHERE #tmp_0_.Rows = #tmpTopRows . Rows 

GROUP BY #tmp_0_. Rows 
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— creating col, grand totals 
SELECT 

Columns, 

CO = SUM {CO ) 

INTO #tmpColumns 
FROM #tmp_0_ 

GROUP BY Columns 

Checking ADJ of GRAND 

SQL: SELECT CO = SUM (CO) INTO #tmpGrand FROM #tmpColumns 

— final results table 

SELECT #tmp_0_.Rows, Columns, CO 

INTO #tmpFinalResults 

FROM #tmp_0_, #tmpTopRows 

WHERE # tmp_0_ . Rows = ttmpTopRows . Rows 

Phase 20: Filtering results ( 841 ) 

Phase 21: Reading Row Totals ( 10 ) 

Phase 22: Reading Column Totals ( 10 ) 

Phase 23: Reading Grand ( 10 ) 

SQL: 

— calculate remaining columns 
SELECT 

Columns = #tmpColumns .Columns, 

CO = #tmpColumns.C0 - ISNULL (SUM ( #tmpFinalResults . CO ) , 0 ) 

INTO #tmpRemaining 

FROM #trapFinalResults , #tmpColumns 

WHERE #tmpColumns .Columns *= #tmpFinalResults . Columns 
GROUP BY #tmpColumns .Columns, ttmpColumns . CO 
select CO = SUM (CO) from #tmpRemaining 

Phase 24: Reading Remaining Column Totals (needed = 10498) ( 20 ) 

Phase 25: Final Results ( 30 ) 

Phase 26: Sorting columns by time (if necessary) . { 20 ) 

Phase 27: sorting rows by time or name (if necessary). { 0 ) ERR FROM BUILD AND EXEC:0 


Getting results took 12658 milliseconds. 

Phase 0: Begining output generation. 

Phase 3: performing cumulative (if necessary). ( 0 ) 

Dollar Amount / Shipped / Gross 
The columns are : 1994 

1995 

1996 

1997 

1998 

The rows are: ******* a number of customer names here ************* 

The table headers are: 

The contents of the results array are: 

***************** 

A number of customer name, value pairs 
***************** 

The contents of the rowtotal array are:< 
key: *****Row totals*****< 


The contents of the coltotal array are: 

1995: ******An amount****** 

1996: ******Ari amount****** 

1997: ******An amount****** 

1998: ******ftn amount****** 

1994: ******ftn amount****** 

The contents of the grdtotal array are: 

Grand: *******a grand total amount******* 

Processing and formatting results took 220 milliseconds. 
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Figure 36 illustrates the options form 3600 that the user can use to select the display options 
for a result. 

Alternative Embodiments 

5 The following describes alternative embodiments of the invention. 

Importantly, various embodiments of the invention do not necessarily include all of the 
% features described above. For example, some embodiments of the invention do not include the 
i j phase of the extraction process (loading the staging tables) because the source system data is 

p provided to the system 1 00 directly by other extraction programs. Another example is where the 

10 datamart 150 is created, but a separate query interface is used to query the datamart 150. The 

ij query interface could use only a different communications protocol (e.g., instead of HTTP), or 
y could be a completely different front end. 

Other embodiments of the invention are configured differently than the embodiments 
described above. For example, the extraction node key in the semantic instance table 308 is not 
15 included, but the extraction node 4 1 0 includes a semantic instance key. 

Some embodiments of the invention build a database system, not necessarily a datamart. 
Additionally, these embodiments do not have to conform to a star schema definition in the 
metadata 160. 

An object database system could be generated instead of a relational database system. 
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Some embodiments of the invention comprise only a computer readable media (e.g., a CD, a 
tape, a hard drive or other storage media) that has the programs that implement all, or a portion 
of, the system 100. Some embodiments of the invention include an electromagnetic waveform 
having the programs. Some embodiments of the invention include only the computer system 
running the datamart, other embodiments of the invention include only the computer system that 
creates, accesses, and queries the datamart, but does not include in the datamart itself. 

Additional, or different, data semantics can be included in other embodiments of the 
invention. 

Some embodiments of the invention include different user interfaces for the enterprise 
manager interface 192 and the query/results interface 184. For example, in the enterprise 
manager interface 192, the semantic types need not be selected in the fact and dimension base 
windows, but can be selected in the semantic instance window. 
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The Claims 


What is claimed is: 

1 . A method of creating a system for creating a well-formed database system using a 
computer, the method comprising: 

the computer accessing a definition of the system, the definition defining a schema for use by 
the system, the schema defining a set of tables, a set of columns that correspond to the set 
of tables, and a set of relationships between the tables of the set of tables, the definition 
further defining a set of operations for manipulating the data, the set of operations 
defining programs that operate on the set of tables and the set of table columns; and 
the computer using the definition to generate the set of tables. 

2. The method of claim 1 wherein the set of tables includes a first table and a second table, 
wherein the first table includes a first column, wherein the second table includes a second 
column, and wherein the first column and the second column are related by a join and are 
therefore guaranteed to be from the same domain. 

3. The method of claim 1 wherein the set of tables includes a first table and a second table, 
and wherein the definition defines that the first table relates to the second table by a many to one 
relationship, and wherein the generating the set of tables includes automatically generating a 
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foreign key column in the first table, wherein the foreign key column is for holding a foreign key 


to the second table. 

4. The method of claim 1 wherein the set of tables includes a first table and a second table, 

and wherein the definition defines that the first table relates to the second table by a many to 
5 many relationship, and wherein the generating the set of tables includes automatically generating 
an associative table corresponding to the first table and the second table, and wherein the 
associative table has a unique value created for each unique many-to-many relationship between 
;= the first table and the second table. 


6. The method of claim 1 wherein the computer using the definition to generate the set of 
tables also includes the computer performing at least some of the set of operations on at least 
some of the set of tables. 

15 7. The method of claim 1 wherein a transaction type column is automatically included in 

some tables of the set of tables. 
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5. The method of claim 1 wherein the set of tables includes a first table and a second table, 
and wherein the first table includes one or more columns from the second table, and wherein said 
one or more columns are automatically populated from the one or more columns. 



8 . 


The method of claim 1 wherein a date column is automatically included in some tables of 


the set of tables. 

9. The method of claim 1 wherein a source system key column is automatically included in 
some tables of the set of tables. 

10. The method of claim 1 wherein the definition defines a set of source system extraction 
operations, wherein the set of source system extraction operations are for extracting data from a 
source system and for manipulating the data for populating the database, and wherein the set of 
source system extraction operations correspond to the schema definition. 

1 1 . The method of claim 1 0 wherein the source system extraction operations correspond to 
the schema definition by populating source system data into the database system according the 
schema definition. 

12. The method of claim 1 wherein the definition defines a set of aggregates for the database 
system, the set of aggregates corresponding to the schema definition, the method further 
comprising: 

the computer using the definition to create a set of aggregate tables corresponding to the set 
of aggregates; and 

populating the set of aggregate tables. 
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1 3 . The method of claim 1 2 wherein the set of aggregates corresponds to the schema 
definition by defining which aggregates should be made from which tables in the database 
system. 

14. The method of claim 12 wherein the definition defines an aggregate operation for an 
aggregate of the set of aggregates. 

15. The method of claim 14 wherein the aggregate operation includes a SUM operation. 

16. The method of claim 14 wherein the aggregate operation includes an AVERAGE 
operation. 

17. The method of claim 1 wherein the definition includes a user interface definition for 
querying the database and for presenting results, the user interface definition corresponding to 
the schema definition. 

1 8. The method of claim 1 7 wherein the user interface definition specifies which columns 
from which tables can be used in a query. 

19. The method of claim 1 wherein the definition defines a set of source system extraction 
operations, a set of aggregates, and a user interface definition, that correspond to the schema 
definition. 

Method and Apparatus for Creating a Well- PATENT Inventors: Craig D. Weissman, 

Formed Database System Using a Computer Page 89 Greg V. Walsh and Eliot L. Wegbreit 

Attorney Docket No 20308 702 ° 

: ODMA\PCDOCS\SQL I \2280 1 7\ I 



20. The method of claim 1 wherein the database system includes a datamart, wherein the 
schema definition includes a star schema definition, wherein the set of tables includes a set of 
fact tables and a set of dimension tables. 

21. A system comprising: 
a database system; 

a first program for accessing a definition of the schema for the database system, the schema 
defining a set of tables, a set of columns corresponding to the set of tables, and a set of 
relationships between the tables of the set of tables, the definition further defining a set of 
operations for manipulating the data, the set of operations defining programs that operate 
on the set of tables and the set of table columns, the first program further for using the 
definition to generate the set of tables. 

22. The system of claim 21 wherein the set of tables includes a first table and a second table, 
wherein the first table includes a first column, wherein the second table includes a second 
column, and wherein the first column and the second column are related by a join and are 
therefore guaranteed to be from the same domain. 

23. The system of claim 21 wherein the set of tables includes a first table and a second table, 
and wherein the definition defines that the first table relates to the second table by a many to one 
relationship, and wherein the generating the set of tables includes automatically generating a 
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foreign key column in the first table, wherein the foreign key column is for holding a foreign key 
to the second table. 

24. The system of claim 21 wherein the set of tables includes a first table and a second table, 
and wherein the definition defines that the first table relates to the second table by a many to 
5 many relationship, and wherein the generating the set of tables includes automatically generating 
an associative table corresponding to the first table and the second table, and wherein the 
associative table has a unique value created for each unique many-to-many relationship between 
n the first table and the second table. 

Lw 25. The system of claim 21 wherein the set of tables includes a first table and a second table, 
iff and wherein the first table includes one or more columns from the second table, and wherein said 

~ one or more columns are automatically populated from the one or more columns. 

g 26. The system of claim 21 wherein the first program includes an enterprise manager for 
accessing the definition, causing the generation of the set of tables, and causing the population of 
the tables. 

15 27. The system of claim 21 further comprising a database, the database for storing the set of 

tables. 
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28. The system of claim 21 further comprising an aggregate building program for accessing a 
definition of a set of aggregates and the definition of the schema and for generating the set of 
aggregates from the definition of the set of aggregates and the definition of the schema. 

29. The system of claim 21 further comprising a query and reporting program for generating 
5 a user interface from a definition of the user interface and the definition of the schema. 

30. A system comprising: 

I*, means for accessing a definition of the system, the definition defining a schema for use by 
a the system, the schema defining a set of tables, a set of columns corresponding to the set 

Ai of tables, and a set of relationships between the tables of the set of tables, the definition 

l|| further defining a set of operations for manipulating the data, the set of operations 

r=; defining programs that operate on the set of tables and the set of table columns; and 

□ means for using the definition to generate the set of tables. 

3 1 . The system of claim 30 wherein the set of tables includes a first table and a second table, 
wherein the first table includes a first column, wherein the second table includes a second 

15 column, and wherein the first column and the second column are related by a join and are 
therefore guaranteed to be from the same domain. 

32. The system of claim 30 wherein the set of tables includes a first table and a second table, 
and wherein the definition defines that the first table relates to the second table by a many to one 
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relationship, and wherein the generating the set of tables includes automatically generating a 
foreign key column in the first table, wherein the foreign key column is for holding a foreign key 
to the second table. 

33. The system of claim 30 wherein the set of tables includes a first table and a second table, 
5 and wherein the definition defines that the first table relates to the second table by a many to 
many relationship, and wherein the generating the set of tables includes automatically generating 
an associative table corresponding to the first table and the second table, and wherein the 

□ associative table has a unique value created for each unique many-to-many relationship between 
0 the first table and the second table. 

10; 34. The system of claim 30 wherein the set of tables includes a first table and a second table, 

and wherein the first table includes one or more columns from the second table, and wherein said 

□ one or more columns are automatically populated from the one or more columns. 

35. The system of claim 34 wherein the definition of the system further includes a definition 
of the aggregates for the system, the system further comprising: 

15 means for generating a set of aggregates from the definition of the aggregates and the 

definition of the schema. 

36. The system of claim 33 wherein the definition of the system further includes a definition 
of the a user interface for the system, the system further comprising: 
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means for generating the user interface from the definition of the user interface and the 
definition of the schema. 

37. The system of claim 33 wherein the definition of the system includes a definition of 
aggregates for use in the system and a definition of a query and reporting mechanism interface 
for the system, the set of tables includes a set of fact tables and a set of dimension tables, and 
wherein the system further comprises: 

means for generating the set of fact tables; 

means for generating the set of dimension tables; 

means for generating a set of aggregate tables; and 

means for generating a query and reporting mechanism interface. 

38. A computer program product comprising: 
a memory medium; and 

a computer program stored on the memory medium, the computer program comprising 
instructions for accessing a definition of a system, the definition defining a schema for 
use by the system, the schema defining a set of tables, a set of columns corresponding to 
the set of tables, and a set of relationships between the tables of the set of tables, the 
definition further defining a set of operations for manipulating the data, the set of 
operations defining programs that operate on the set of tables and the set of table 
columns, and instructions for using the definition to generate the set of tables. 
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39 The computer program product of claim 38 wherein the set of tables includes a first table 
and a second table, wherein the first table includes a first column, wherein the second table 
includes a second column, and wherein the first column and the second column are related by a 
join and are therefore guaranteed to be from the same domain. 

5 40. The computer program product of claim 38 wherein the set of tables includes a first table 

and a second table, and wherein the definition defines that the first table relates to the second 
table by a many to one relationship, and wherein the generating the set of tables includes 
3 automatically generating a foreign key column in the first table, wherein the foreign key column 
is for holding a foreign key to the second table. 

lb; 41 . The computer program product of claim 38 wherein the set of tables includes a first table 
=; and a second table, and wherein the definition defines that the first table relates to the second 
□ table by a many to many relationship, and wherein the generating the set of tables includes 
0 automatically generating an associative table corresponding to the first table and the second 

table, and wherein the associative table has a unique value created for each unique many-to-many 
15 relationship between the first table and the second table. 

42. The computer program product of claim 38 wherein the set of tables includes a first table 
and a second table, and wherein the first table includes one or more columns from the second 
table, and wherein said one or more columns are automatically populated from the one or more 
columns. 
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43. A computer data signal embodied in a carrier wave comprising: 

a computer program, the computer program comprising instructions for accessing a definition 
of a system, the definition defining a schema for use by the system, the schema defining a 
set of tables, a set of columns corresponding to the set of tables, and a set of relationships 
5 between the tables of the set of tables, the definition further defining a set of operations 

for manipulating the data, the set of operations defining programs that operate on the set 
of tables and the set of table columns, and instructions for using the definition to generate 
the set of tables. 

■=; 44. The computer data signal embodied in the carrier wave of claim 43 wherein the set of 

l§ tables includes a first table and a second table, wherein the first table includes a first column, 

a wherein the second table includes a second column, and wherein the first column and the second 
column are related by a join and are therefore guaranteed to be from the same domain. 

^ 45 . The computer data signal embodied in the carrier wave of claim 43 wherein the set of 

tables includes a first table and a second table, and wherein the definition defines that the first 
15 table relates to the second table by a many to one relationship, and wherein the generating the set 
of tables includes automatically generating a foreign key column in the first table, wherein the 
foreign key column is for holding a foreign key to the second table. 

46. The computer data signal embodied in the carrier wave of claim 43 wherein the set of 
tables includes a first table and a second table, and wherein the definition defines that the first 
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table relates to the second table by a many to many relationship, and wherein the generating the 
set of tables includes automatically generating an associative table corresponding to the first 
table and the second table, and wherein the associative table has a unique value created for each 
unique many-to-many relationship between the first table and the second table. 

47. The computer data signal embodied in the carrier wave of claim 43 wherein the set of 
tables includes a first table and a second table, and wherein the first table includes one or more 
columns from the second table, and wherein said one or more columns are automatically 
populated from the one or more columns. 
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The Abstract 


A method of defining a well-formed database system by defining the organization of the data 
in the database, and by defining the operations for that data, is described. The definition can be 
used to automatically create and populate the well-formed database system. The well-formed 
5 database system conforms to rules of correctness and produces results that conform to the rules. 
The organization is defined by a data organization definition that specifies tables, their columns, 
and the relationships between tables. The operations define procedures that operate on the tables 
and the table columns. Importantly, the operations are defined along with the tables, columns, 
and relationships, so that the resulting system is well-formed. 
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Appendix A 

Appendix A illustrates semantic types that may be supported and their corresponding 
adaptive template names. For example, the Pipelined semantic type is made up of, in this order, 
the mapjceys the pipe_state and the index_fact adaptive templates. The example pre-parsed and 
5 post parsed SQL adaptive templates are then provided. 

As mentioned previously, the use of the semantic types significantly reduces the amount of 
work needed to implement the datamart 150. By selecting a semantic type for a particular fact 
:=: table or dimension table, the consultant automatically selects the corresponding pre-parsed SQL 

p adaptive templates. The selected adaptive templates are then automatically converted into post 

parsed SQL statements that include the schema specific information for the datamart 1 50. 
fl Additionally, these post parsed SQL statements include the SQL for accessing and manipulating 
the datamart 150 tables. 




Pipelined 

|map_keys i 

Pipelined 

|pipe_state ; 

^Pipelined 

iindex_fact j 

Pipelined/Unjoined 

|upd_unj ' 

Pipelined/Unjoined 

map_keys 1 

Pipelined/Unjoined 

Pipeiined/Unjoined 

|pipe_state ; 

'indexjact , 


[Slowly Changing Dimensions 'H®®^-^' 1 ?! 

[Slowly Changing Dimensions index_dim 


Transactional 

imap_keys 

i 

Transactional 

|load_trans 


Transactional 

|ren_trans 

! 

Transactional 

iindex_fact 

i 

— —I 

jTransactional/Inventory 

imap_keys 

] 
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i T ransactional/l n ventory 

iload_trans 

i 

Transactional/Inventory 

|inv_adjust 

; 

'T ransactional/lnventory 

iindex_fact 

1 

i 

;Transactional/l nventory/ForceZero 

map_keys 


; T ransactional/l nventory/ForceZero 

loadjrans 

i 

Transactional/I nventory/ForceZero 

jforce zero 

— ■ — ■ T r J . n . 

< 

jT ransactional/l nventory/ForceZero 

'inv_adjust 

j 

. ..j 

! T ransactional/l nventory/ForceZero 

!index_fact 



I T ransactional/l n ventory/ForceZero/U njoined [ upd_unj 


Transactional/Inventory/ForceZero/Unjoined |map_keys 


|Transactional/lnventory/ForceZero/Unjoined 

1 

'load trans 

i ~ 

| 

i 


i 

,Transactional/l nventory/ForceZero/U njoined 

jforce_zero 


i 

ITransactional/Inventory/ForceZero/Unjoined 

iinv_adjust 


i 


Transactional/I nventory/ForceZero/U njoined .index Jfact 


Transactional/I nventory/U njoined 

jupd_unj 


) 

iTransactional/Inventory/Unjoined 

;map_keys 


| 

1 T ransactional/l nventory/U njoined 

!load_trans 


j 

i T ransactional/l nventory/U njoined 

;inv_adjust 


• 


lfransactional/1 n ventory/U njoined ~iindex_fact 

! Transactional/Stateiike j map_keys 


[T ransactional/Statelike Joad^rans 


Transactional/Stateiike lload state 



| ] 


Transactional/Statelike/ForceClose/Unjoined 

i 

i ... _ . . 

!load_trans i 

! 1 

!Transactional/Statelike/ForceClose/Unjoined 

jforce_close j 

j 

J 

| T ransactional/Statelike/ForceClose/Unjoined 

iload_state i 
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ITransactional/Statelike/ForceClose/Unjoined 

j 

index_fact i 

J 1 

Transactional/Statelike/Unjoined 

;upd_unj 

i T ransactional/Statelike/Unjoined 

imap_keys 

'Transactional/Statelike/Unjoined 

!load_trans 

jTransactional/Statelike/Unjoined 

{Transactional/Statelike/Unjoined 

,Transactionai/Unjoined 

!load_state i 

|index_fact 

jupd_unj i 

Transactional/Unjoined 

map_keys • 

{Transactional/Unjoined 

load_trans 

jTransactional/Unjoined 

jren_trans j 

jTransactional/Unjoined 

index fact 

”* j 


The following are the pre-parsed pseudo-SQL source for the adaptive templates. 


— #TEMPLATE_BEGIN# forceclose 

— Copyright * 1997, Epiphany Marketing Software, Inc. All Rights Reserved. 

— force_close 

— Close out deleted orders - those that no longer appear in the 

— staging table 

— SEE SAFETY VALVE BELOW 

I**************************************************-****************-*************#**/ 

/*********★********•*■★******•***★**★■*•*****★*★★***★**•**★*★★****★★******★•**★•**•»********/ 

— Delete temporary tables 

y**********************************************************************-*******'*****y 

— #BLOCK_BEGIN# DropTemps 

$$DDL_BEGIN 

$$DROP JFABLE_IF_EXISTS [$$FCTTBL [ ] _FC] 

$$DDL_END 

— #BLOCK_END# DropTemps 

^^^★★★^★^★★★★♦★★★★★★★★★★★★★★★★★★★★★★★★★★^********-*****-**-* , *Tt*'*-'***'*'***-*'**-******-*-***-** y 

— Insert negative BOOKs for deleted orders 

— FC: Force Close 

y************** + *************************************:**********************-***--*'* , **fc* y 

— #BLOCK_BEGIN# MakeFC 

$ $ S ELECT_INTO_BEGIN [ $ $ FCTTBL [ ] _FC ] 

SELECT 

f .iss, 
f . ss_key, 

MAX (f . date_key) date_key, 

MIN (f. transtype key) transtype key, 
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MAX(f.seq) + 1 
f .$$DIMKEYR_01 
f .$$DIMKEYR_02 
f .$$DIMKEYR__03 
f ,$$DIMKEYR_04 
f .$$DIMKEYR__05 
f .$$DXMKEYR_06 
f ,$$DIMKEYR_07 
f .$$DIMKEYR_08 
f .$$DIMKEYR_09 
f .$$DIMKEYRJL0 
f . $ $ DEGKEY_0 1 
f . $ $ DEGKEY_0 2 
f . $$DEGKEY 03 


-SUM ( f . $ $ FCTCOLJDO 1 ) 
-SUM ( f . $$FCTCOL_002) 

- SUM ( f . $ $ FCTCOL_0 0 3 ) 
-SUM ( f . $$FCTCOL_004 ) 
-SUM ( f . $$FCTCOL_005) 

- SUM ( f . $ $ FCTCOL_0 0 6 ) 
-SUM ( f . $ $ FCTCOL_007 ) 
-SUM ( f . $$FCTCOL_008 } 
-SUM ( f . $$FCTCOL_009) 
-SUM { f . $ $ FCTCOL__0 1 0 ) 
- SUM ( f . $ $ FCTCOL_0 1 1 ) 
-SUM ( f . $ $ FCTCOL_0 1 2 ) 
-SUM ( f . $ $ FCTCOL_0 1 3 ) 
- SUM ( f . $ $ FCTCOL_0 1 4 ) 
-SUM (f . $$FCTCOL_015) 
-SUM (f . $ $ FCTCOL_0 1 6 ) 
-SUM ( f . $$ FCTCOL_017 ) 
- SUM ( f . $ $ FCTCOLj) 1 8 ) 
-SUM { f . $ $ FCTCOL_0 1 9 ) 
-SUM (f . $$FCTCOL_020 ) 
- SUM ( f . $ $ FCT COL_0 2 1 ) 
-SUM { f . $$FCTCOL_022 ) 
- S UM ( f . $ $ FCTCOLJ) 2 3 ) 
- SUM ( f . $ $ FCTCOLJ) 2 4 ) 


$ $ FCTCOL_0 0 1 
$ $ FCTCOL_0 0 2 
$$FCTCOL_003 
$$FCTCOL_004 
$ $ FCTCOL_0 0 5 
$ $ FCTCOL_0 0 6 
$$FCTCOL_007 
$ $ FCTCOL_0 0 8 
$$FCTCOL_009 
$$FCTCOL_010 
$ $ FCTCOL_0 1 1 
$$FCTCOL_012 
$$FCTCOL_013 
$ $ FCTCOL_0 1 4 
$ $ FCTCOL_0 1 5 
$ $ FCTCOL_0 1 6 
$ $ FCTCOL_0 1 1 
$$FCTCOL_018 
$ $ FCTCOL_0 1 9 
$ $ FCTCOL_0 2 0 
$ $ FCTCOL_02 1 
$$FCTCOL_022 
$$FCTCOL_023 
$ $ FCTCOL_02 4 


$$SELECT_INTO_BODY [ $$FCTTBL ( ] _FC] 

FROM 

$$FCTTBL[] $$CURR f 

WHERE 

NOT EXISTS 

(SELECT 1 FROM $ $FSTGTBL [ ] _MAP S WHERE S . iss 

GROUP BY 

f . iss, 
f .ss_key 
f .$$DIMKEYR_01 
f ,$$DIMKEYR_02 
f ,$$DIMKEYR_03 
' f .$$DIMKEYR_04 

f .$$DIMKEYR_05 
' f .$$DIMKEYR_06 

f .$$DIMKEYR_07 
f .$$DIMKEYR_08 
f .$$DIMKEYR_09 
f .$$DIMKEYR_X0 
f . $$DEGKEY_01 
f . $$DEGK£Y_02 
f . $ $ DEGKE Y_0 3 


= f.iss AND s.ssjcey = f.ss_key) 


HAVING 


OR 

OR 

OR 

OR 

OR 


( SUM ( f . $ $ FCTCOL_0 0 1 ) <> 0) 
(SUM(f . $$FCTCOL_002 ) <> 0) 
(SUM ( f . $$FCTCOL_Q03 ) <> 0) 
( SUM ( f . $ $ FCTCOL_0 0 4 ) <> 0) 
(SUM ( f . $$FCTCOL_005) <> 0) 
( SUM ( f . $ $ FCTCOL 006) <> 0) 
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OR 

( SUM ( f . $ $ FCTCOL 007) 

<> 

0) 

OR 

( SUM ( f . $ $ FCTCOL 008) 

<> 

0) 

OR 

(SUM ( f . $$FCTCOL_009) 

<> 

0) 

OR 

( SUM ( f . $ $ FCTCOL 010) 

<> 

0) 

OR 

( SUM ( f . $ $ FCTCOL Oil) 

<> 

0) 

OR 

( SUM ( f . $ $ FCTCOL_0 12 ) 

<> 

0) 

OR 

( SUM ( f . $ $ FCTCOL_0 1 3 ) 

<> 

0) 

OR 

( SUM ( f . $ $ FCTCOL_0 1 4 ) 

<> 

0) 

OR 

(SUM ( f . $ $ FCTCOL 015) 

<> 

0) 

OR 

( SUM ( f . $ $ FCTCOL 016) 

<> 

0) 

OR 

( SUM ( f . $ $ FCTCOL 017) 

<> 

0) 

OR 

( SUM ( f . $ $ FCTCOL 018) 

<> 

0) 

OR 

(SUM ( f . $$FCTCOL__019 ) 

<> 

0) 

OR 

( SUM { f . $ $ FCTCOL 020) 

<> 

0) 

OR 

( SUM ( f . $ $ FCTCOL 021) 

<> 

0) 

OR 

( SUM ( f . $ $ FCTCOL 022) 

<> 

0) 

OR 

(SUM (f.$$ FCTCOL 023) 

<> 

0) 

OR 

( SUM { f . $ $ FCTCOL__02 4 ) 
\ 

<> 

0) 

i 

AND 


MIN ( f . transtype_key) 

<= 

99 

AND 


MIN ( f . transtype_key) 

>= 

1 


— #BLOCK_END# MakeFC 

,*****. ****************************************************************************'' 
SAFETY VALVE - THIS PROC ONLY DOES ANYTHING 

;;*^.I^*^**i*™************-— ****™********************************"***** / 

— #BLOCK_BEGIN# SafetyValue 

DECLARE $$VAR[count_MAP] $$EPIINT$$EOS 

BEGIN 

$$VAR_ASSIGN_BEGIN[count_MAPj 
SELECT COUNT (1) 

$ $VAR_ASSIGN_INTO [ count_MAP] 

FROM $ $ FSTGTBL [ ] _MAP 
$ $VAR_ASSIGN_END 

$SIF[ ($$VAR[count_MAP] = 0)] 

DELETE FROM $$FCTTBL [ ]_FC$$EOS 
$$END_IF 

END$$EOS 

— #BLOCK_END# SafetyValue 

/,**.***, **»********************************************************************** v 
;;.^LS”“”«Li“*^*^..**.....**.**«**********************************/ 

— #BLOCK__BEGIN# SPResultS 
BEGIN 

INSERT INTO adaptive_template_prof ile (token name, numbe r _ r °w s ) 

SELECT 'PROCESSED', COUNT (1) FROM $$FCTTBL [ ] $$CURR$$EOS 

INSERT INTO adaptxve_template __profile ( tokenjriame, number__rows) 

SELECT ’INSERTED', COUNT(l) FROM $$FCTTBL [ ]_FC$$EOS 

END$$EOS 

— #BLOCK_END# SPResults 
— #TEMPLATE_END# force_close 
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— #TEMPLATE_BEGIN# load_state 

^***********************************************************************************/ 

— Copyright * 1997, Epiphany Marketing Software, Inc. All Rights Reserved. 

— load_state 

— Load order bookings into fact table by creating transactional 

— data from state data 

— load_trans must be run before this procedure to create TIN table 

y**************************************************'*********************************/ 

y**********+*******************-******+**********************************************/ 

— Delete temporary tables 

y************************* fr*********************************************************/ 

— #BLOCK_BEGIN# DropTemps 
$$DDL_BEGIN 

$ $ DROP_TABLE_I F_EX I STS [ $ $ FCTTBL [ ]_MFL] 

$$DROP_TABLE_IF_EXISTS [ $$FCTTBL [ ] 1ST ] 

$ $ DROP_TABLE_I F_EXI STS [ $ $ FCTTBL [ ] _I L ] 

$$DROP_TABLE_IF_EXISTS [$$ FCTTBL [ ] X R] 

$ $ DROP _TABLE_I F_EX I STS [ $ $ FCTTBL [ ] _IRD] 

$ $ DROP_TABLE_I F_EXISTS [$$ FCTTBL [ ] _IND] 

$ $ DROP_TABLE_I F_EXI STS [ $ $ FCTTBL [ ] _NFD ] 

$$DROP_TABLE_IF_EXISTS [$$ FCTTBL [ ]_IRM] 

$ $DROP_TABLE_IF_EXI STS [$$ FCTTBL [ ]_IDM] 

$ $ DROP_TABLE_I F_EXI STS [ $ $ FCTTBL [ ] _I LM ] 

$ $ DROP_TABLE_I F_EXI STS [ $$ FCTTBL [ ] _IMI ] 

$$DDL_END 

— #BLOCK_END# DropTemps 

^******************************************************************y 

— Set join order for SQL Server 

/**********★**********************★********************************/ 

— #BLOCK_BEGIN# ForcePlanOn 
$$ SQLSERVER [SET FORCEPLAN ON] 

— #BLOCK_END# ForcePlanOn 

y***********************************************************************************/ 

— Remove rows older than fact table - history can not be rewritten - only 

— the last date for an order can be changed. Note that we compare transtype's 

— because SHIP type transactions might occur at a later date and we don’t want 

— those to interfere 

— Also, since the staging table may have multiple entries for a given order on 

— a single day - we assume that the list one inserted in the Staging table will 

— be used (since ikey is an IDENTITY column) 

-- Note that a given ss_key must use the same Booking transtype for all of time, 

-- otherwise the transtype_key 

— MFL : Mapped Filtered 

/*********★★******★****★*******************★****************************************/ 

--#BLOCK_BEGIN# MakeMFL 

$$SELECT_INTO_BEGIN[$$FCTTBL[]_MFL] 

SELECT 

s . * 

$ $ SELECT_INTO_BOD Y [ $ $ FCTTBL [ ] _MFL ] 

FROM 
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WHERE 


$$FSTGTBL [ ]_MAP s, bus_process b 

((s.date_key >= (SELECT MAX (date_key) FROM $$FCTTBL [ ] $$CURR f WHERE 
s.iss = f.iss AND s.ss_key = f.ss_key AND 
s.transtype_key = f . transtype_key) ) 

OR NOT EXISTS (SELECT * FROM $$FCTTBL [ ] $$CURR f WHERE 
s.iss = f.iss AND s . ss_key = f.ss_key AND 
s.transtype_key = f . transtype_key) ) 
s.ikey = (SELECT MAX(t.ikey) FROM $ $ FSTGTBL [ ] _MAP t WHERE 
s.iss = t.iss AND 
s.ss_key = t.ss_key AND 

s. date_key = t.date_key AND 

t . process_key = b. process key) 

AND “ 

s .process_key = b.process_key AND b .process_name = ' LoadState* 

— #BLOCK_END# MakeMFL 


Index MFL table for later queries 


/ 

/ 


— #BLOCK_BEGIN# IndexMFL 

$$DDL_BEGIN 
$$DDL_EXEC [ 

CREATE INDEX X$$FCTTBL[] MFL ON $$FCTTBL [ ] MFL 

( 

iss, ss_key, date_key 

) 

] 

$$DDL END 


— # BLOCK_END# IndexMFL 


/********************************* ************************************** *********^ + 
Get oldest state rows for each unique sskey 


/ 


-- We need to treat the first entry for each order 

in the staging table separately from all others, since 

— only the first entry needs to be compared with 
already existing fact entry rows to create transactions. 

— All subsequent dates for that order in the Fact table 

— can be delta* d with other staging table entries - see the 

— section below on Pairwise deltas. 


— MFL should be indexed 


1ST: The first record for each iss, ss_key 
/*************** ********************************** ****^^^ # ^^^^^^^^ 


— #BLOCK BEGIN# MakelST 


$$SELECT__INTO_BEGIN[$$FCTTBL[] 1ST] 

SELECT 

S.* 

$ $ SELECT_INTO_BODY [ $ $ FCTTBL [ ] 1ST ] 

FROM 

$$ FCTTBL []_MFL s 

WHERE 

s.date_key = (SELECT MIN (date_key) FROM $$ FCTTBL [ ]_MFL t WHERE 
s.iss = t.iss AND s.ss_key = t.ss_key) 

— #BLOCK END# MakelST 








— Index 1ST for later queries 
/*******************************************^^^^^ 

— #BLOCK_BEGIN# IndexlST 




/ 

/ 
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$$DDL_BEGIN ' 

$ $ DDL_EXEC [ 

CREATE UNIQUE INDEX XPK$$FCTTBL [ ]_1ST ON $$FCTTBL [ ] 1ST 
iss, ss_key 

) 

] 

$$DDL_END 

#BLOCK_END# IndexlST 

— Insert negative BOOKs for changed dim keys 

— This query will add up all existing Books and Loss's 

— for this order and the net facts will be cancelled out 

— with the old Dimension keys. Note that an invariant of this 
procedure is that only one set of dimensions at a time 

— can have non-zero facts. 

Fact table Should be indexed 

HAVING Clause is needed to prevent changing of dimensions 
on fully shipped order from causing a transaction - no sense 
creating fact rows with all zero's m them 

Note that we increment the sequence number just in case 
-- this new transaction occurs on the same date as the last 
-- existing one in the fact table - to avoid index errors 

— IL: InsertLost 

, 

— #BLOCK_BEGIN# MakelL 

$ $ SELECT_INTO_BEGIN [ $ $ FCTTBL [] IL] 

SELECT 

s .iss, 
s . ss_key, 
s . date_key, 
s . transtype_key, 

MAX(f.seq) + 1 seq 
f .$$DIMKEYR_01 
f .$$DIMKEYR_02 
f .$$DIMKEYR_03 
f .$$DIMKEYR_04 
f .$$DIMKEYR_05 
f .$$DIMKEYR_06 
f .$$DIMKEYR_07 
f .$$DIMKEYR_08 
f .$$DIMKEYR_09 
f .$$DIMKEYR_10 
f . $ $ DEGKEY_0 1 
f . $$DEGKEY_02 
f . $$DEGKEY 03 


-SUM ( f , 
-SUM ( f , 
-SUM ( f , 
-SUM ( f . 
-SUM ( f . 
-SUM ( f . 
-SUM ( f . 
-SUM ( f . 
-SUM ( f . 
-SUM ( f . 
-SUM ( f . 
-SUM ( f . 
-SUM ( f . 
-SUM { f . 
-SUM ( f . 


. $$FCTCOL 
,$$FCTCOL‘ 
. $$FCTC0L* 
.$$FCTCOL _ 

,$$fctcol" 
$ $ fctcol" 

$$FCTCOL~ 

$$FCTCOL~ 

$$FCTCOL~ 

$$FCTCOL_ 

$$FCTCOL_ 

$$FCTCOL_ 

$$FCTC0L_ 

$$FCTCOL_ 

$$FCTCOL 


001 ) 

002 ) 

"003) 

004) 

"005) 

"006) 

"007) 

008) 

009) 

010 ) 

011) 

012) 

013) 

014) 

015) 


$$FCTCOL_001 
$$FCTCOL_002 
$$FCTCOL_003 
$ $ FCTCOL_0 0 4 
$$FCTCOL_005 
$ $ FCTCOL_0 0 6 
$$FCTCOL_007 
$$FCTCOL_008 
$$FCTCOL_009 
$$FCTCOL_010 
$ $ FCTCOL_0 1 1 
$$FCTCOL_012 
$$FCTCOL_013 
$$FCTCOL_014 
$$FCTCOL 015 
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/ 

- SUM ( f . $ $ FCTCOL 

016) 

$$ FCTCOL 016 

f 

- SUM ( f . $ $ FCTCOL 

017) 

$$ FCTCOL 017 

t 

- SUM ( f . $ $ FCTCOL 

018) 

$$ FCTCOL 018 

t 

- SUM ( f . $ $ FCTCOL 

019) 

$$ FCTCOL 019 

t 

-SUM ( f . $ $ FCTCOL 

020) 

$$ FCTCOL 020 

t 

-SUM ( f . $ $ FCTCOL 

021) 

$$ FCTCOL 021 

t 

-SUM ( f . $ $ FCTCOL 

022) 

$$ FCTCOL 022 

t 

-SUM ( f .$$ FCTCOL 

023) 

$$ FCTCOL 023 

t 

-SUM ( f .$$ FCTCOL 

024) 

$$ FCTCOL 024 

$$ SELECT INTO BODY [$$FCTTBL [ ] IL] 

FROM 






$$FCTTBL [ ] 1ST £ 

», $ $ FCTTBL [ ] $ $ CURR f 

WHERE 






s.iss = f.iss AND s.ss key = f.ss_key 

AND 






( {s . $$DIMKEYR 06 <> 

e,$$DIMKEYR 06) OR 


(S.$$DIMKEYR 05 

<> f 

. $$DIMKEYR 05) OR 


( s . $$DIMKEYR 07 

<> f 

. $$DIMKEYR 07) OR 


( s . $$DIMKEYR 04 

<> f 

. $$DIMKEYR 04) OR 


( s . $$DIMKEYR 08 

<> f 

. $$DIMKEYR 08) OR 


{ s . $$DIMKEYR 03 

<> f 

. $$DIMKEYR 03) OR 


(S.$$DIMKEYR 09 

<> f 

. $$DIMKEYR 09) OR 


( s . $$DIMKEYR 02 

<> f 

. $$DIMKEYR 02) OR 


( s . $$DIMKEYR 10 

<> f 

. $$DIMKEYR 10) OR 


(S.$$DIMKEYR 01 

<> f 

. $$DIMKEYR 01) ) 

GROUP 

BY 





s . iss, 





s . ss_key, 
s.date_key, 
s. transtype key 




/ 

f . $$DIMKEYR 01 




/ 

/ 

f . $$DIMKEYR 02 
f . $$DIMKEYR 03 




/ 

f . $$DIMKEYR 04 




r 

r 

f . $$DIMKEYR 05 
f . $$DIMKEYR 06 




, 

f.$$DIMKEYR 07 




r 

f.$$DIMKEYR 08 




r 

f . $$DIMKEYR 09 




, 

f . $$DIMKEYR 10 




r 

f . $ $ DEGKE Y 01 




r 

f . $ $ DEGKE Y 02 




t 

f ,$$DEGKEY_03 




HAVING 

MIN { f . transtype_ 

_key) 

_ 

3 . transtype_key 

AND 

( 

{SUM(f .$$ FCTCOL 

001) 

<> 

0) 

OR 

{SUM{f .$$ FCTCOL 

002) 

<> 

0) 

OR 

(SUM (f .$$ FCTCOL 

003) 

<> 

0) 

OR 

( SUM ( f . $ $ FCTCOL 

004) 

<> 

0) 

OR 

(SUM(f .$$ FCTCOL" 

"005) 

<> 

0) 

OR 

( SUM ( f . $ $ FCTCOL" 

"006) 

<> 

0) 

OR 

( SUM ( f . $ $ FCTCOL" 

"007) 

<> 

0) 

OR 

( SUM ( f . $ $ FCTCOL" 

"008) 

<> 

0) 

OR 

( SUM ( f . $ $ FCTCOL" 

"009) 

<> 

0) 

OR 

( SUM ( f . $ $ FCTCOL" 

"010) 

<> 

0) 

OR 

(SUM(f .$$ FCTCOL" 

"Oil) 

<> 

0) 

OR 

(SUM (f.$$ FCTCOL" 

"012) 

<> 

0) 

OR 

( SUM ( f . $ $ FCTCOL* 

013 ) 

<> 

0) 

OR 

(SUM(f .$$ FCTCOL" 

“014) 

<> 

0) 

OR 

(SUM ( f . $$FCTCOL“ 

"015) 

<> 

0) 

OR 

( SUM ( f . $ $ FCTCOL* 

"016) 

<> 

0) 

OR 

{ SUM ( f . $ $ FCTCOL" 

"017) 

<> 

0) 

OR 

( SUM ( f . $ $ FCTCOL" 

"018) 

<> 

0) 

OR 

{ SUM ( f . $ $ FCTCOL" 

"019) 

<> 

0) 

OR 

{ SUM { f . $ $ FCTCOL" 

"020) 

<> 

0) 

OR 

( SUM ( f . $ $ FCTCOL* 

~021) 

<> 

0) 

OR 

(SUM (f .$$ FCTCOL" 

"022) 

<> 

0) 
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OR 

OR 


{ SUM ( f . $$FCTCOL__023) <> 0) 
(SUM ( f . $$FCTCOL_024 ) <> 0) 


— #BLOCK_END# MakelL 

— Index IL for later queries ******************************************** 


— # BLOCK BEGIN# IndexIL 


f 

/ 


$$DDL_BEGIN 
$ $ DDL_EXEC [ 

CREATE INDEX XPK$$FCTTBL [ ]_IL ON $$FCTTBL [ ] _IL 
( 

iss, ss_key 

) 

] 

$ $DDL_END 


— # BLOCK END# IndexIL 


y**************************************************** 

— insert BOOKS for changed dim keys 




— When a dimension changes then just create a booking 

— transaction for whatever we negated above with the new 

— dimension and fact values 


— 1ST shoud be indexed 

-- Note that we add one to whatever we used as the last 

— seq because this transaction occurs on the same 

— date as the negative one above 


— IR: Insert Rebook 
y******************************** 




★ * 




— #BLOCK BEGIN# MakeIR 


$ $ SELECT_I NTO_BEGI N [ $ $ FCTTBL [ ] IR] 

SELECT 


s . iss, 
s ,ss_key, 
s .date_key, 

1 . transtype_key, 
l.seq + 1 seq 
s . $$DIMKEYR_01 
S . $$DIMKEYR_02 
s . $ $ DIMKEYR_03 
s.$$DIMKEYR_04 
S . $ $ DIMKEYR_0 5 
s.$$DIMKEYR_06 
S.$$DIMKEYR_07 
S.$$DIMKEYR_08 
S . $$DIMKEYR__09 
s . $$DIMKEYR_10 
s . $$DEGKEY_01 
S . $$DEGKEY_02 
s * $$DEGKEY_03 

-1 . $ $ FCTCOL_0 0 1 
-l.$$FCTCOL_002 
-1 . $$FCTCOL_003 
-l.$$FCTCOL_004 
-l.$$FCTCOL_005 
- 1 . $ $ FCTCOL_0 0 6 
- 1 . $ $ FCTCOL_0 0 7 
- 1 . $ $ FCTCOL_0 0 8 
-1 . $$FCTCOL 009 


$ $ FCTCOL_0 0 1 
$$FCTCOL_002 
$$FCTCOL_003 
$$FCTCOL_004 
$$FCTCOL_005 
$$FCTCOL_006 
$$FCTCOL_007 
$ $ FCTCOL_0 0 8 
$$FCTCOL 009 
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1 . $$FCTCOL_010 
1 . $ $ FCTCOL_0 1 1 
1 . $$FCTCOL_012 
■l.$$FCTCOL_013 
•1 . $ $ FCTCOL_0 1 4 
l.$$FCTCOL_015 
1 . $ $ FCTCOL__0 1 6 
■1.$$FCTC0LJ)17 
■1 . $$FCTCOL_018 
•1 . $ $ FCTCOL_0 1 9 
1 . $$FCTCOL__020 
1 . $$FCTCOL_021 
1 . $ $ FCTCOL_02 2 
•1 . $$FCTCOL_023 
1 . $$FCTCOL_024 


$$FCTCOL_010 
$ $ FCTCOL_0 1 1 
$$FCTCOL_012 
$ $ FCTCOL_0 1 3 
$$FCTCOL_014 
$ $ FCTCOL_0 1 5 
$$FCTCOL_016 
$$FCTCOL_017 
$ $ FCTCOL_0 1 8 
$$FCTCOL_019 
$$FCTCOL_020 
$ $ FCTCOL_02 1 
$$FCTCOL_022 
$ $ FCTCOL_0 2 3 
$$FCTCOL 024 


$$SELECT_INTO_BODY [$$FCTTBL [ ]_IR] 

FROM 

$$FCTTBL [ ]_IL 1, $$FCTTBL [ ]_1ST s 
WHERE l.iss = s.iss AND l.ss_key = s.ss_key 

— #BLOCK_END# MakeIR 

/*************★★***********★**************★*★********************************+******/ 

— Insert BOOKS for changed dim keys where fact 
-- also changed 

— When a dimension changes at the same time as 

— a fact then we need to make up the fact difference 


— 1ST shoud be indexed 


— Note that we add two to whatever we used as the last 

— seq because this transaction occurs on the same 

— date as the negative and positive ones above 

— Note also that the Left Outer join uses transtype_key 

— so that only the Bookings at the old value will be counted. 

— Whereas above for the negative transaction value 

— we want to include Shipments in our calculation, here 

— we only want to see how Booking Facts have changed. 

— Here again, only one Booking transaction type is supported 

— per ss_key 

— IRD: Insert Rebook delta 

/****+********★************★******************★★************★***********************/ 
— #BLOCK_BEGIN# MakeIRD 


$$SELECT_INTO_BEGIN[$$FCTTBL[]_IRD] 

SELECT 


s . iss, 
s . ss_key, 
s .date_key, 
s . transtype_key, 
l.seq + 2 seq 
s.$$DIMKEYR_01 
s . $ $ DIMKEYR_02 
s . $$DIMKEYR_03 
S.$$DIMKEYR_04 
S.$$DIMKEYR_05 
s . $$DIMKEYR__06 
S . $ $ DIMKEYR_07 
S.$$DIMKEYR_08 
s . $$DIMKEYR_09 
s . $$DIMKEYR_10 
s . $ $ DEGKEY_0 1 
S . $ $ DEGKEY_02 
s . $$DEGKEY 03 
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MAX (s . 
MAX ( S . 
MAX ( s . 
MAX (s . 
MAX (s . 
MAX (s . 
MAX {s . 
MAX ( S . 
MAX (s . 
MAX ( S . 
MAX (s . 
MAX ( S . 
MAX ( S . 
MAX ( S . 
MAX ( s . 
MAX ( s . 
MAX ( S . 
MAX (S . 
MAX {s . 
MAX ( s . 
MAX ( s . 
MAX ( S . 
MAX (S 
MAX {s 


$$FCTCOL_ 
$$FCTCOL_ 
$ $ FCTCOL_ 
$$ FCTCOL 
$$ fctcol" 

$$FCTCOL_ 
$$FCTCOL_ 
$$FCTCOL_ 
$$ FCTCOL 
$$ fctcol" 

, $$FCTCOL_ 
, $$ FCTCOL~ 
^FCTCOL' 
.$$FCTCOL' 

,$$fctcol" 
.$$fctcol" 
. $$ fctcol" 
.$$fctcol" 

. $$ FCTCOL 
. $$ FCTCOL" 
. $$ FCTCOL 
. $$ FCTCOL 
. $$ FCTCOL 
. $ $ FCTCOL 


001 ) 

002 ) 

003) 

004) 

005) 

006) 

007) 

008) 
’009) 
’ 010 ) 
"Oil) 
" 012 ) 
"013) 
’014) 
’015) 
’016) 
'017) 
"018) 
'019) 
_ 020 ) 
_ 021 ) 

022 ) 

"023) 

024) 


-$$NVL [ SUM ( f . 
-$$NVL [SUM ( f . 
-$$NVL [SUM ( f . 
-$$NVL [SUM { f . 
-$$NVL[SUM(f . 
-$$NVL [ SUM ( f . 
-$$NVL [ SUM { f . 
-$$NVL [ SUM ( f . 
-$$NVL [SUM ( f . 
-$$NVL [SUM { f . 
-$$NVL [ SUM ( f . 
-$$NVL [ SUM ( f . 
-$$NVL [ SUM ( f . 
-$$NVL [SUM (f . 
-$$NVL [SUM ( f . 
-$$NVL [ SUM ( f . 
-$$NVL [ SUM ( f . 
-$$NVL[SUM(f . 
-$$NVL [SUM (f 
-$$NVL [SUM ( f 
-$$NVL [SUM (f 
-$$NVL [ SUM ( f 
-$$NVL[SUM(f 
-$$NVL [ SUM ( f 


$ $ FCTCOL_ 
$$FCTCOL_ 
$$ FCTCOL' 
$$FCTCOL_ 
$$FCTCOL_ 
$$ FCTCOL 
$$ FCTCOL' 
$$ FCTCOL" 
$$ FCTCOL" 
^FCTCOL" 

.$$fctcol" 

,$$fctcol" 

.$$fctcol" 

. $$ FCTCOL - 
.$$FCTCOL’ 

.$$ fctcol" 
.$$fctcol" 

.$$ FCTCOL" 
.$$ FCTCOL 
. $$ FCTCOL 
. $$ FCTCOL 
.$$ FCTCOL 
.$$ FCTCOL 
.$$ FCTCOL 


001 ) 

002 ) 

003) 

004) 

005) 

006) 
007) 
"008) 
'009) 
" 010 ) 
"Oil) 
" 012 ) 
’013) 
’014) 
’015) 
’016) 
"017) 
"018) 
~019) 
_ 020 ) 
_ 021 ) 
_ 022 ) 

023) 

"024) 


0) 

0] 

0] 

-X- 0] 

— 0] 
0] 
0 ] 
0 ] 
0 ] 

— 0 ] 
0 ] 
0 ] 
0 ] 
0 ] 
0 ] 
0 ] 
0 ] 
0 ] 
0 ] 
0 ] 
0 ] 
0 ] 
0 ] 
0 ] 


$$FCTCOL_001 
$$FCTCOL_002 
$$FCTCOL_003 
$ $ FCTCOL__00 4 
$$FCTCOL_005 
$$FCTCOL_006 
$$FCTCOL__007 
$ $ FCTCOL_0 0 8 
$$FCTCOL_009 
$$ FCTCOL JH0 
$ $ FCTCOL_0 1 1 
$ $ FCTCOL_0 1 2 
$ $ FCTCOL_0 1 3 
$ $ FCTCOL_0 1 4 
$ $ FCTCOL_0 1 5 
$$FCTCOL_016 
$ $ FCTCOL_0 1 7 
$ $ FCTCOL_0 1 8 
$ $ FCTCOL_0 1 9 
$$FCTCOL_020 
$ $ FCT COL_0 2 1 
$$FCTCOL_022 
$$FCTCOL_023 
$ $ FCTCOL_0 2 4 


IRD] 


s.iss = f.iss AND s.ss_key = f.ssjcey AND 


$ $ SELECT_INTO_BODY [ $ $ FCTTBL [ ] 

FROM 

$$ FCTTBL [ ] _IL 1, $$ FCTTBL []_1 ST S 
$ $ LO J_FROM [ $ $ FCTTBL [ ] $ $ CURR f 
s . transtype_key = f . transtype_key] 

WHERE 

l.iss = s.iss AND l.ss key = s.ss_key 

$$JOIN_WHERE[s. iss - f.iss (+) AND s.ss_key = f.ss_key {+) AND s.trans ype_ ey 
f . transtype_key { + ) 1 
GROUP BY 

S . ISS, 

s . ss_key, 
s.date_key, 
s . transtype__key, 

1 . seq 

s .$$DIMKEYR_01 
s.$$DIMKEYR_02 
S . $$DIMKEYR_Q3 
s.$$DIMKEYR_04 
S.$$DIMKEYR_05 
s . $ $ DIMKEYR__0 6 
S.$$DIMKEYR_07 
S.$$DIMKEYR_08 
s . $$DIMKEYR_09 
S.$$DIMKEYR_10 
S . $ $ DEGKE Y_0 1 
s.$$DEGKEY_02 
S.$$DEGKEY_03 


HAVING 

($$NVL [SUM ( f . $$FCTCOL 001) 


OR 

($$NVL [SUM ( f .$$ FCTCOL 002) 

~ 

OR 

($ $NVL ( SUM (f.$$ FCTCOL 003) 

~ 

OR 

( $ $NVL [ SUM ( f . $ $ FCTCOL 004) 


OR 

( $ $NVL [ SUM ( f . $ $ FCTCOL_0 0 5 ) 


OR 

($$NVL [SUM ( f . $$ FCTCOL 006) 


OR 

( $ $ NVL [ S UM ( f . $ $ FCTCOL_0 0 7 ) 

~ f 

OR 

($ $NVL [ SUM (f.$$ FCTCOL 008) 

~ ~ 

OR 

( $ $N VL [SUM ( f - $ $ FCTCOL 009) 

"* r 

OR 

( $$NVL [SUM ( f . $$FCTCOL_010) 

~ ~ 

OR 

( $ SNVL [ SUM ( f . $ $ FCTCOL_0 1 1 ) 


OR 

( $$NVL [SUM ( f . $$FCTCOL_012 ) 


OR 

($$NVL[SUM(f .$$FCTCOL_013) 

~ , ~ 

OR 

( $ $N VL [ SUM ( f . $ $ FCTCOL 014) 

~ , 

OR 

OR 

( $ $NVL [ SUM ( f . $ S FCTCOL_01S) 
($$NVL[SUM(f .$$ FCTCOL 016) 

~ r ~ 


<> MAX ( S . 
<> MAX {s . 
<> MAX ( S . 
<> MAX ( S . 
<> MAX { s . 
<> MAX (s . 
<> MAX ( S . 
<> MAX (s . 
<> MAX ( s . 
<> MAX ( s . 
<> MAX (s . 
<> MAX (S . 
<> MAX (s . 
<> MAX (s . 
MAX ( S . 
MAX ( S . 


<> 

<> 


$$FCTCOL_ 
$$ FCTCOL 
$$ FCTCOL" 

$$fctcol" 

$$fctcol" 

$$fctcol" 

$$ FCTCOL" 
$$ FCTCOL' 
$$ FCTCOL" 
$ $ FCTCOL" 
$$ FCTCOL 
$$ FCTCOL 
$$ FCTCOL 
, $ $ FCTCOL 
.$$ FCTCOL 
. $$ FCTCOL 


001 ) ) 
002 } ) 

003) ) 

004) ) 

005) ) 
"006) ) 
'007) ) 
008) ) 
"009) ) 
’010) ) 

011) ) 

012 ) ) 
013) ) 

"014) ) 
015) ) 
'016)) 
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;;{i !jjt 


OR 

OR 

OR 

OR 

OR 

OR 

OR 

OR 


( $$NVL [SUM ( f 
( $$NVL [ SUM ( f 
( $$NVL [ SUM ( f 
{$$NVL [SUM { f 
( $$NVL [SUM (f 
($$NVL [SUM ( f 
($$NVL [SUM { f 
($$NVL [ SUM ( f 


,$$FCTCOL 
,$$FCTCOL 
,$$ FCTCOL 
,$$ FCTCOL 
, $$FCTCOL 
, $$FCTCOL 
. $$FCTCOL 
. $ $ FCTCOL 


017) 

"018) 

"019) 

' 020 ) 

" 021 ) 

" 022 ) 

"023) 

"024) 


0] <> MAX(s.$$FCTCOL_017) ) 
0] <> MAX(s.$$FCTCOL_018) ) 
0] <> MAX (S. $$FCTCOL_019) ) 
0] <> MAX {s . $$FCTCOL_020) ) 
0] <> MAX ( s . $ $ FCTCOL_02 1 ) ) 
0] <> MAX (s . $$FCTCOL_022 ) ) 
0] <> MAX(S.$$FCTCOL_023) ) 
0] <> MAX(s.$$FCTCOL_024) ) 


— #BLOCK_END# MakeIRD 

^********************* ************************* *************************************/ 

— Insert BOOKS for deltas with same dim keys OR for 

— brand new orders. 

— Note that we DON'T want to count Shipments 

-- (so shipment ss_key's should be different from 

— order ss keys) since we just want bookings to sum up 

— to whatever this transcation says they should be. 

— Fact table should be indexed 

-- WHERE clause prevents double booking on changed 

— dimension - if we didn't use the NOT EXISTS clause 

— then this query would repeat the work of the last one 

— above - which we have already taken care of 

— HAVING clause ensures that multiple 0 records don't 

— get inserted whenever this procedure is run 

— Note that we increment the sequence number just m case 

— this new transaction occurs on the same date as the last 

— existing one in the fact table - to avoid index errors 

/*******i**** ************************************** *********************************/ 

— # BLOCK_BEGI N # MakeIND 

$$SELECT_INTO_BEGIN [ $$FCTTBL [ ] _IND] 

SELECT 

s . iss, 
s . ss_key, 
s .date__key, 
s . transtype_key, 

$$NVL [MAX ( f . seq) 0] + 1 seq 

, s.$$DIMKEYR_01 

S.$$DIMKEYR_02 
S.$$DIMKEYR_03 
S.$$DIMKEYR_04 
s . $ $ DIMKEYR_0 5 
S . $ $ DIMKEYR_0 6 
S.$$DIMKEYR_07 
S.$$DIMKEYR_08 
S.$$DIMKEYR_09 
S.$$DIMKEYR_10 
s.$$DEGKEY_01 
s . $ $ DEGKE Y__0 2 
s.$$DEGKEY_03 


MAX (s 
MAX ( s 
MAX ( S 
MAX ( S 
MAX ( S 
MAX (s 
MAX ( S 
MAX (s 
MAX (S 
MAX (S 
MAX (S 


$ $ FCTCOL_0 0 1 ) 
$ $ FCTCOL_0 0 2 ) 
$ $ FCTCOL_0 0 3 ) 
$ $ FCTCOL_0 0 4 ) 
$$FCTCOL_005) 

, $ $ FCTCOL_0 0 6 ) 
, $$FCTCOL_007 ) 
, $ $ FCTCOL_0 0 8 ) 
. $ $ FCTCOL_0 0 9 ) 
. $ $ FCTCOL_0 1 0 ) 
.$$ FCTCOL Oil) 


-$$NVL [ SUM ( f 
-$$NVL [SUM ( f 
-$$NVL [SUM ( f 
-$$NVL [SUM { f 
-$$NVL [SUM ( f 
-$$NVL [SUM ( f 
-$$NVL [SUM ( f 
-$$NVL [SUM ( f 
-$$NVL [SUM ( f 
-$$NVL [SUM ( f 
-$$NVL [SUM ( f 


$$ FCTCOL 
.$$ FCTCOL” 
, $$FCTCOL_ 
, $ $ FCTCOL_ 
, $ $ FCTCOL_ 
, $$FCTCOL 
.$$FCTCOL _ 
.$$ FCTCOL" 
. $$FCTCOL_ 
.$$ FCTCOL" 

,$$fctcol" 


001 ) 

002 ) 

003) 

004) 

005) 

006) 

007) 

008) 

009) 

010 ) 
Oil) 


0] 

0 ] 

0] 

0] 

0] 

0 ] 

0 ] 

0 ] 

0 ] 

0 ] 

0 ] 


$ $ FCTCOL_0 0 1 
$$FCTCOL_002 
$ $ FCTCOL_0 0 3 
$ $ FCTCOL_0 0 4 
$ $ FCTCOL_0 0 5 
$ $ FCTCOL_0 0 6 
$ $ FCTCOL_0 0 7 
$$FCTCOL_008 
$$FCTCOL_009 
$$FCTCOL_010 
$$ FCTCOL Oil 
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MAX { S . 
MAX ( S . 
MAX ( s . 
MAX { S . 
MAX ( S . 
MAX ( S . 
MAX {S . 
MAX(S. 
MAX (S. 
MAX ( S . 
MAX { S . 
MAX ( S . 
MAX (S . 


$$ FCTCOL 

$$fctcol" 

$$FCTCOL _ 

$$ fctcol" 

$$FCTCOL 
$$ FCTCOL" 

$$fctcol' 
$$ fctcol] 

$ $ FCTCOL 
$$ FCTCOL 
$$ FCTCOL 
$$ FCTCOL 
$$ FCTCOL 


012 )- 

013) ' 

014) 

015) 

016) 

017) 

018) 
'019) 
" 020 ) 
“ 021 ) 
" 022 ) 
"023) 

024 ) 


-$$NVL [SUM (f . 
-$$NVL [ SUM ( f . 
-$$NVL [SUM ( f . 
-$$NVL [SUM {f . 
-$$NVL [SUM (f . 
-$$NVL[SUM(f . 
-$$NVL[SUM(f . 
-$$NVL [SUM (f . 
-$$NVL [SUM ( f . 
-$$NVL [ SUM ( f . 
-$$NVL [SUM (f . 
-$$NVL[SUM{f . 
-$$NVL[SUM{f . 


$$FCTCOL_ 
$$ FCTCOL" 
$$ FCTCOL' 

$$fctcol" 

$$ FCTCOL" 
$$ FCTCOL] 
$$ FCTCOL] 
$$ FCTCOL' 
$$ FCTCOL 
$$ FCTCOL 
$$ FCTCOL 
$$ FCTCOL 
$$ FCTCOL 


012 ) 
"013) 
"014) 
"015) 
"016) 
"017) 
"018) 
"019) 
" 020 ) 
" 021 ) 
" 022 ) 
_023 ) 
024) 


0] 

0] 

01 

0] 

0] 

0] 

0] 

01 

0 ] 

0 ] 

0 ] 

0 ] 

0 ] 


$$FCTCOL_012 
$$FCTCOL_013 
$ $ FCTCOL_0 1 4 
$ $ FCTCOL__0 1 5 
$ $ FCTCOL_0 1 6 
$$FCTCOL_017 
$$FCTCOL_018 
$$FCTCOL_019 
$$FCTCOL__020 
$$FCTCOL_021 
$ $ FCTCOL_0 2 2 
$ $ FCTCOL_0 2 3 
$$FCTCOL__024 


$ $ SELECT_INTO_BODY [ $ $ FCTTBL [ ] _INDl 

FROM _ ^ 

$$ FCTTBL [ ] 1ST S $$LOJ FROM [ $$FCTTBL [ ] $$CURR f 

s.iss = f.iss AND s.ss_key = f.ssjcey AND f . transtype_key - s . transtype_key] 

WHERE NOT EXISTS (SELECT * FROM $ $ FCTTBL [ ] _IL WHERE iss = s.iss AND ss_key = S.SS_key) 

$$ JOIN WHERE [s .iss = f.iss (+) AND s.ss_key = f.ss_key (+) AND s.transtype_key 


f . transtype_key 
GROUP BY 


( + )] 


, 

s.iss, 
s . ss_key, 
s .date_key, 
s. transtype key 
s.$$DIMKEYR 01 

/ 

S . $$DIMKEYR 02 

, 

S . $$DIMKEYR 03 


S . $$DIMKEYR 04 


S.$$DIMKEYR 05 


s . $$DIMKEYR 06 


s . $$DIMKEYR 07 


s . $$DIMKEYR_08 


s . $$DIMKEYR__09 


S.$$DIMKEYR_10 

t 

S . $$DEGKEY 01 

t 

s . $$DEGKEY 02 

, 

s . $ $ DEGKEY_0 3 

HAVING 

OR 

($$NVL [SUM ( f . $$FCTCOL 001) 
( $ $N VL [ SUM ( f . $ $ FCTCOL_0 0 2 ) 

OR 

( $ $NVL [ SUM ( f . $ $ FCTCOL_0 0 3 } 

OR 

( $ $NVL [ SUM ( f . $ $ FCTCOL_0 0 4 ) 

OR 

( $$NVL [ SUM ( f . $$ FCTCOL 005) 

OR 

( $ $ NVL [ SUM { f . $ $ FCTCOL_0 0 6 ) 

OR 

{$$NVL [SUM ( f . $$ FCTCOL_0 0 7 ) 

OR 

($$NVL [SUM ( f . $$FCTCOL 008) 

OR 

($$NVL [ SUM ( f . $$ FCTCOL 009) 

OR 

( $ $ NVL [ SUM ( f . $ $ FCTCOL_0 1 0 ) 

OR 

($$NVL [SUM { f . $$ FCTCOL Oil) 

OR 

( $ $ NVL [ S UM ( f . $ $ FCTCOL_0 1 2 ) 

OR 

{ $ $ NVL [ SUM ( f . $ $ FCTCOL_0 1 3 ) 

OR 

( $ $NVL [ SUM { f . $ $ FCTCOL_0 1 4 ) 

OR 

( $ $ NVL [ SUM ( f . $ $ FCTCOL_0 1 5 ) 

OR 

($$NVL [ SUM ( f . $$FCTCOL_016) 

OR 

( $ $ NVL [ SUM ( f . $ $ FCTCOL_0 1 7 ) 

OR 

( $ $ NVL [ SUM ( f . $ $ FCTCOL_0 18 ) 

OR 

{ $$NVL [SUM ( f . $$ FCTCOL 019) 

OR 

($$NVL [SUM ( f . $$ FCTCOL 020) 

OR 

($$NVL [ SUM ( f . $$FCTCOL_021 ) 

OR 

( $ $NVL [ SUM ( f . $ $ FCTCOL_0 22 ) 

OR 

( $$NVL [SUM ( f .$$ FCTCOL 023) 

OR 

( $ $NVL [ SUM { f . $ $ FCTCOL 024) 


0] <> 
0] 

0] 

01 
0] 

01 
0] 

0] 

0] 

0] 

0] 

01 <> 
0 ] <> 
<> 
<> 
<> 


<> 

<> 

<> 

<> 

<> 

<> 

<> 

<> 

<> 

<> 


01 

0] 

0] 


- 01 <> 
- 0] <> 
- 0] <> 
- 0] <> 
- 0] <> 
- 0] <> 
,-01 <> 
01 <> 


MAX ( s . 
MAX ( s . 
MAX ( s , 
MAX {s < 
MAX { s < 
MAX(S, 
MAX ( s , 
MAX ( s 
MAX ( s 
MAX {S 
MAX (s 
MAX {s 
MAX ( s 
MAX ( s 
MAX { S 
MAX (s 
MAX (s 
MAX {s 
MAX (s 
MAX ( s 
MAX ( S 
MAX (s 
MAX (s 
MAX (S 


$ $ FCTCOL_ 
$$FCTCOL_ 
$$FCTCOL_ 
$$FCTCOL_ 
. $ $ FCTCOL_ 
, $ $ FCTCOL_ 
. $ $ FCTCOL" 
. $$FCTCOL_ 
. $$ FCTCOL' 
. $$ FCTCOL 

.$$fctcol“ 

. $$ FCTCOL 

,$$fctcol - 
. $$fctcol" 
.$$fctcol" 
. $$ fctcol" 
,$$fctcol| 
. $$ fctcol] 
.$$ fctcol" 
.$$ fctcol" 
.$$ fctcol" 
,$$fctcol" 
,$$fctcol" 
.$$ fctcol" 


001 )) 
002 ) ) 

003) ) 

004) ) 

005) ) 

006) ) 

007) ) 

008) ) 
'009} ) 
' 010 ) ) 
"Oil)) 
" 012 )) 

013) ) 
‘014) ) 

015) ) 

016) ) 
017)) 

'018) ) 
019)) 
_ 020 ) ) 
_ 021 ) ) 
_ 022 ) ) 
023) ) 
”024) ) 


--#BLOCK END# MakeIND 
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— Form pairwise deltas for all rows except earliest for each sskey 

— Each row created in NFD will consist of two sequential entries from the 

— stainq table. So if N enties for an order exist in MFL (after we have filtered 

— out same-date duplicates) then all the queries above will deal with the earliest entry, 

-- 6 all S the queries below (including this one) will deal with the N-l deltaing transactions 

-- This query assumes that MFL will already have been filtered 

— to have a single record for each order/datekey 

7;***********i******i****** *********************************************************/ 

— #BLOCK_BEGIN# MakeNFD 

$ $ SELECT_INTO_BEGIN [ $ $ FCTTBL [ ] _NFD] 

SELECT 

s.iss siss, t.iss tiss 
, s.ss_key sss_key, t.ssjcey tss_key 

s.date key sdate_key, t.date_key tdate_key 

s. transtype key stranstype_key, t . transtype_key ttranstype_key 
s . $ 5DIMKEYR 01 s$$DIMKEYR_01, t . $$DIMKEYR_01 t$$DIMKEYR_01 
S $$DIMKEYR 02 s$$DIMKEYR_02 , t . $$DIMKEYR_02 t$$DIMKEYR_02 
s!$$DIMKEYR“03 s $ $ DI MKE YR_0 3 , t . $$DIMKEYR_03 t$$DIMKEYR_03 
s . $$DIMKEYR 04 s$$DIMKEYR_04 , t . $$DIMKEYR_04 t $ $ D IMKE Y R_0 4 
S.$$DIMKEYR 05 s$$DIMKEYR_05, t . $$DIMKEYR_05 t$$DIMKEYR_05 
S . $$DIMKEYR _ 06 s$$DIMKEYR_06, t . $$DIMKEYR_06 t$$DIMKEYR_06 
S $$DIMKEYR 07 s$$DIMKEYR_07 , t . $$DIMKEYR_07 t$$DIMKEYR_07 
s.$$DIMKEYR 08 s$$DIMKEYR_08 , t . $$DIMKEYR_08 t$$DIMKEYR_08 
S.$$DIMKEYR 09 s$$DIMKEYR_09, t . $$DIMKEYR_09 t$$DIMKEYR_09 
s*$$DIMKEYR~10 s $ $ DIMKEYR_1 0 , t . $$DIMKEYR_10 t$$DIMKEYR_10 
s ! $$DEGKEY 01 s$$DEGKEY_01 , t . $ $ DEGKEY_0 1 t$$DEGKEY_01 
s . $$DEGKEY 02 s$$DEGKEY_02 , t . $ $ DEGKEY_0 2 t$$DEGKEY_02 
s . $$DEGKEY 03 s$$DEGKEY_03, t . $ $ DEGKEY_0 3 t $ $ DEGKE Y_0 3 
S . $ $ FCTCOL 001 s $ $ FCTCOL_0 01, t . $$FCTCOL_001 t$$FCTCOL_001 
s . $$FCTCOL 002 s$$FCTCOL_002 , t . $$FCTCOL_002 t$$FCTCOL_002 
s . $$ FCTCOL 003 s$$FCTCOL__003, t . $$FCTCOL_003 t$$FCTCOL_003 
s . $$ FCTCOL 004 s$$FCTCOL_004 , t . $ $ FCTCOL_0 0 4 t$$FCTCOL_004 
s . $ $ FCTCOL 005 S$$FCTCOL_005, t . $$FCTCOL_005 t$$FCTCOL_005 
s $ $ FCTCOL - 0 0 6 s$$FCTCOL__006, t . $$FCTCOL_006 t$$FCTCOL_006 
S.$$FCTCOL"007 s$$FCTCOL_007, t . $$FCTCOL_007 t$$FCTCOL_007 
S $$ FCTCOL 008 s$$FCTCOL_008, t . $$FCTCOL_008 t$$FCTCOL_008 
s!$$FCTCOL"009 s $ $ FCTCOL_0 0 9 , t . $$FCTCOL_009 t$$FCTCOL_009 
S . $$ FCTCOL 010 s $ $ FCTCOL_0 1 0 , t . $$FCTCOL_010 t$$FCTCOL_010 
s!$$FCTCOL _ 011 s$$FCTCOL_011, t .$$FCTCOL_011 t$$FCTCOL_011 
S $$ FCTCOL 012 s $ $ FCTCOL_0 12, t . $$FCTCOL_012 t$$FCTCOL_012 
s . $ $ FCTCOL - 0 1 3 s $ $ FCTCOL_0 13, t . $$FCTCOL_013 t$$FCTCOL_013 
S . $$FCTCOL 014 s $ $ FCTCOL_0 1 4 , t . $$FCTCOL_014 t$$FCTCOL_014 
S.$$ FCTCOL 015 s$$FCTCOL_015, t . $$FCTCOL_015 t$$FCTCOL_015 
S . $$FCTCOL _ 016 S $ $ FCTCOL_0 1 6 , t . $$FCTCOL_016 t$$FCTCOL_016 
S . $$ FCTCOL 017 s $ S FCTCOL_0 17 , t . $$FCTCOL_017 t$$FCTCOL__017 
S.$$FCTCOL _ 018 s$$FCTCOL_018, t . $$FCTCOL_018 t$$FCTCOL_018 
s.$$ FCTCOL 019 s $ $ FCTCOL_0 19, t . $$FCTCOL_019 t$$FCTCOL_019 
s ! $ $ FCTCOL - 02 0 s$$FCTCOL_020, t . $$FCTCOL_020 t$$FCTCOL_020 
S.$$ FCTCOL 021 s$$FCTCOL_021, t . $$FCTCOL_021 t$$FCTCOL_021 
s . $$ FCTCOL 022 s$$FCTCOL_022 f t . $$FCTCOL_022 t$$FCTCOL_022 
s . $$ FCTCOL 023 s$$FCTCOL_023, t . $$FCTCOL_023 t$$FCTCOL_023 
s ] $ $ FCTCOL_0 2 4 s$$FCTCOL_024 , t . $$FCTCOL_024 t$$FCTCOL_024 

$ $ SELECT_INTO_BODY [$$ FCTTBL []_NFD] 

FROM 


WHERE 


AND 


$ $ FCTTBL [] _MFL s, $$ FCTTBL [] _MFL t 
s.iss = t.iss AND s.ss_key = t.ss_key 

s.date key = (SELECT MAX (date_key) FROM $$ FCTTBL []_MFL u WHERE 
u.iss = s.iss AND u.ss_key = s.ss_key AND u.date_key < t.date_key) 


— #BLOCK END# MakeNFD 


Method and Apparatus for Creating a Well- PATENT 
Formed Database System Using a Computer Page 1 1 3 

Attorney Docket No 20308 702 
ODMAXPCDOC S\SQL l\228017\l 


Inventors: Craig D. Weissman, 
Greg V. Walsh and Eliot L. Wegbreit 


/ 


fc***************************^ 7 ^”' 




— Insert BOOKS for deltas with same dim keys 

— If the dimensions don't change then we create a 

— new booking order (as long as at least one of the facts 

— have changed) 


— I DM: Insert Delta More 


— #BLOCK_BEGIN# Make I DM 

$ $ SELECT_INTO_BEGI N [ $ $ FCTTBL [ ] _I DM] 

SELECT 

tiss iss, 
tss_key ss_key, 
tdate_key date__key, 
ttranstypejcey transtype_key, 

0 seq 

, t$$DIMKEYR_01 $$DIMKEYR_01 

, t$$DIMKEYR_02 $$DIMKEYR_02 

, t$$DIMKEYR_03 $$DIMKEYR_03 

f t$$DIMKEYR_04 $$DIMKEYR_04 

, t$$DIMKEYR_05 $$DIMKEYR_05 

, t$$DIMKEYR__06 $$DIMKEYR__06 

, t$$DIMKEYR_07 $$DIMKEYR_07 

, t $ S DIMKE YR_0 8 $$DIMKEYR_08 

, t$$DIMKEYR_09 $$DIMKEYR_09 

, t$$DIMKEYR_JLO $$DIMKEYR_10 

, t$$DEGKEY__01 $$DEGKEY_01 

, t$$DEGKEY_02 $$DEGKEY_02 

, t$$DEGKEY_03 $$DEGKEY_03 

t $ $ FCTCOL_0 0 1 - s $ $ FCTCOL_0 01 $ $ FCTCOL__00 1 
' t $ $ FCTCOL _ 0 0 2 - S $ $ FCT COL_0 02 $ $ FCTCOL_0 0 2 

t$$FCTCOL 0 0 3 - S $ $ FCTCOL_0 0 3 $$FCTCOL_003 
' t$$FCTCOL 0 0 4 - S $ $ FCTCOL_0 0 4 $$FCTCOL_004 

t $ $ FCTCOL_0 0 5 - s $ $ FCTCOL_0 05 $ $ FCT COL _0 0 5 

t$$FCTCOL_006-s$$FCTCOL_006 $$FCTCOL_006 
' t $ $ FCTCOL_0 0 7 - S $ $ FCTCOL_0 07 $ $ FCTCOL__0 0 7 

' t $ $ FCTCOL_0 0 8 - s $ $ FCTCOL_0 08 $ $ FCTCOL_0 0 8 

' t $ $ FCTCOL_0 0 9 - S $ $ FCTCOL_0 0 9 $ $ FCTCOL_0 0 9 

t $ $ FCTCOL_0 1 0 - s $ $ FCTCOL_0 1 0 $$FCTCOL_010 
t$$FCTCOL”011-s$$FCTCOL_011 $$FCTCOL_011 
t $ $ FCTCOL_0 1 2 - s $ $ FCTCOL_0 12 $ $ FCTCOL_0 1 2 
t$$FCTCOL 0 1 3 - s$ $ FCTCOL_0 1 3 $$FCTCOL_013 
t$$FCTCOL 0 1 4 - s $ $ FCTCOL_0 1 4 $$FCTCOL_014 
t$ $ FCTCOL_0 1 5-S $ $ FCTCOL_0 1 5 $$FCTCOL_015 
t$$FCTCOL_016-s$$ FCTCOL__016 $$FCTCOL_016 
t $ $ FCTCOL _ 0 1 7 - s $ $ FCTCOL_0 1 7 $$FCTCOL_017 
t S $ FCTCOL _ 0 1 8 - S $ $ FCTCOL_0 1 8 $$FCTCOL_018 
t $ $ FCTCOL_0 1 9- s $ $ FCTCOL_0 1 9 $ $ FCTCOL_0 1 9 
' t $ $ FCTCOL_0 2 0 - s $ $ FCTCOL_0 20 $ $ FCTCOL_0 2 0 

t $ $ FCTCOL_0 2 1 - s $ $ FCTCOL_0 2 1 $$FCTCOL_021 
t $ $ FCTCOL_0 2 2 - s $ $ FCTCOL_0 22 $ $ FCTCOL_02 2 
t $ $ FCTCOL_0 2 3 - S $ $ FCTCOL_0 2 3 $$FCTCOL_023 
' t $ $ FCTCOL_0 2 4 - s $ $ FCTCOL_0 24 $ $ FCTCOL_0 2 4 

$ $SELECT_INTO_BODY [ $$ FCTTBL [ ] _IDM] 

FROM 

$$ FCTTBL []_NFD d 

WHERE 

( S $ $ DIMKE YR_0 6 = t$$DIMKEYR_06) AND 
(s$$DIMKEYR_05 = t$$DIMKEYR__05 ) AND 
(S$$DIMKEYR_07 = t$$DIMKEYR_07 ) AND 
( s $ $ DIMKE YR_0 4 = t $ $ DIMKEYR_0 4 ) AND 
( S $ $ DIMKEYR_0 8 = t$$DIMKEYR_08 ) AND 
(s$$DIMKEYR 03 - t$$DIMKEYR 03) AND 
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( S $ $ DIMKE YR_0 9 = t$$DIMKEYR_09) AND 
(s$$DIMKEYR_02 = t$ $ DIMKEYR_0 2 ) AND 
( S$$ DIMKE YR__10 = t$$DIMKEYR_10) AND 
(S$$DIMKEYR_01 = t$ $ DIMKE YR_0 1 ) 

) 


AND 


OR 

OR 

OR 

OR 

OR 

OR 

OR 

OR 

OR 

OR 

OR 

OR 

OR 

OR 

OR 

OR 

OR 

OR 

OR 

OR 

OR 

OR 

OR 


( 

(s$$FCTCOL_ 
( S$$FCTCOL_ 
{ s$$FCTCOL_ 
(s$$FCTCOL_ 
( s$$FCTCOL_ 
(s$$FCTCOL_ 
( s$$FCTCOL 
(s$$FCTCOL _ 
{s$$fctcol“ 

(S$$FCTCOL - 
(s$$FCTCOL _ 
(s$$FCTCOL“ 
( S$$ FCTCOL" 
(s$$ fctcol' 
(s$$ FCTCOL' 
( s$$FCTCOL 
(s$$ FCTCOL" 
( s$$ fctcol" 
(s$$FCTCOL 
( s $ $ FCTCOL 
( s$$FCTCOL 
(s$$FCTCOL 
(s$$FCTCOL 
{s$$ FCTCOL 
) 


001 <> 
'002 <> 

003 <> 

004 <> 

005 <> 
'006 <> 
'007 <> 
'008 <> 


009 <> 

010 <> 

011 <> 

012 <> 
'013 <> 
*014 <> 
'015 <> 
*016 <> 
'017 <> 
'018 <> 
019 <> 
"020 <> 
"021 <> 
"022 <> 
"023 <> 
"024 <> 


t$$FCTCOL_ 
t$$FCTCOL 
t$$FCTCOL' 
t$$FCTCOL_ 
t$$FCTCOL_ 
t$$FCTCOL_ 
t$$ FCTCOL" 
t$ $ FCTCOL" 
t$$FCTCOL _ 
t$$FCTCOL" 
t$$FCTCOL" 
t$$FCTCOL" 
t$$FCTCOL" 
t$ $ FCTCOL" 
t$ $ FCTCOL" 
t$$ FCTCOL" 
t$$ FCTCOL" 
t$$ FCTCOL" 
t$$ FCTCOL' 
t$$ FCTCOL 
t$$FCTCOL 
t$$ FCTCOL 
t$$FCTCOL 
t$$FCTCOL 


001 ) 

002 ) 

003) 

004) 

005) 

006) 
'007) 
"008) 
"009) 
" 010 ) 
"Oil) 
" 012 ) 
"013) 
"014) 
- 015) 
- 016) 
"017) 
_018) 
_CH9) 
_ 020 ) 

021 ) 
_ 022 ) 
_023) 
024) 






— #BLOCK_END# MakelDM 
^***. + **************************-* ****** 

— Insert negative BOOKS for deltas with different dim keys 

-- If one of the dimensions change then we first create a lose transaction for 

— all the previous facts. (Negate all the facts from the earlier of the two 

— transactions) 


— ILM: Insert Lost More 










— #BLOCK_BEGIN# Make ILM 

$ $ SELECT_INTO_BEGIN [ $ $ FCTTBL [ ] _ILM] 
SELECT 

siss iss, 
sss_key ss_key, 
tdate_key date_key, 
stranstype_key transtypejcey, 
0 seq 

, s$$DIMKEYRj)l $ $ DIMKEYR_0 1 

S$$DIMKEYR_02 $$DIMKEYR_02 
s$$DIMKEYR_03 $$DIMKEYR_03 
S$$DIMKEYR_04 $$DIMKEYR_04 
S$$DIMKEYR_05 $$DIMKEYR_05 
s$$DIMKEYR_06 $$DIMKEYR_06 
sS$DIMKEYR_07 $$DIMKEYR_07 
s$$DIMKEYR_08 $$DIMKEYR_08 
s$$DIMKEYR_09 $$DIMKEYR_09 
S$$DIMKEYR_10 $ $ DIMKEYR_1 0 
s $ $ DEGKE Y_0 1 $ $ DEGKEY_0 1 
S$$DEGKEY_02 $$DEGKEY_02 
S $ $ DEGKEY_0 3 $ $ DEGKEY_0 3 

-s$$ FCTCOL 001 $ $ FCTCOL 001 
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-S$$FCTCOL_ 

-s$$FCTCOL_ 

-S$$FCTCOL 

-S$$FCTCOL“ 

-s$$FCTCOL _ 

-s$$fctcol" 

-S$$FCTC0L” 

-s$$fctcol" 
-s$ $ fctcol" 
-s$$ fctcol" 
-s$ $ fctcol" 

-s$$ FCTCOL' 

-s$$ fctcol' 

-s$$FCTCOL_ 
-s$$FCTCOL~ 
-S$$FCTCOL 
-s$$ FCTCOL* 

-s$ $ fctcol" 

-S$$ FCTCOL 
-S$$FCTCOL 
-s$$FCTCOL 
-S$$ FCTCOL 
-S$S FCTCOL 


002 $ $ FCTCOL_0 02 
'003 $$FCTCOL_003 
'004 $ $ FCTCOL__0 0 4 
'005 $ $ FCTCOL_0 0 5 
'006 $ $ FCTCOL_0 0 6 
'007 $$FCTCOL_007 

008 $$FCTCOL_008 

009 $ $ FCTCOL_0 0 9 
"010 $$FCTCOL_010 
'Oil $ $ FCTCOL_0 1 1 
'012 $ $ FCTCOL_0 1 2 
"013 $$FCTCOL_013 
"014 $ $ FCTCOL_0 1 4 
"015 $ $ FCTCOL_0 1 5 

016 $ $ FCTCOL_0 1 6 
”017 $ $ FCTCOL_0 1 7 
~018 $$FCTCOL_018 
019 $ $ FCTCOL_0 1 9 
”020 $$FCTCOL_020 
”021 $ $ FCTCOL_0 2 1 
022 $ $ FCTCOL_0 2 2 
”023 $ $ FCTCOL__02 3 
024 $ $ FCTCOL_0 2 4 


$$SELECT_INTOJBODY [ $$FCTTBL [ ] _ILM] 
FROM 

$ $ FCTTBL [ ] _NFD d 


( s $ $ DIMKEYR_0 6 <> t$$DIMKEYR_06) OR 
( s $ S DIMKEYR_0 5 <> t$$DIMKEYR_05) OR 
( s $ $ DIMKEYR_0 7 <> t$$DIMKEYR_07) OR 
(s$$DIMKEYR_04 <> t$$DIMKEYR_04 ) OR 
(S$$DIMKEYRJ)8 <> t$$DIMKEYR_08 ) OR 
( S$$DIMKEYR_03 <> t$$DIMKEYR_03 ) OR 
(s$$DIMKEYR_09 <> t $ $ DIMKEYR_0 9 ) OR 
{S$$D1MKEYR_02 <> t $ $ DIMKE YR_0 2 ) OR 
(s$$DIMKEYR_10 <> t$$DIMKEYR_10) OR 
(S$$ DIMKE YR_01 <> t$$DIMKEYR_01 ) 


( S $ $ FCTCOL_0 0 1 <> 0) 

( S $ $ FCTCOL_0 0 2 <> 0) 

( S $ $ FCTCOL_0 0 3 <> 0) 
(s$$FCTCOL_004 <> 0) 
(s$$FCTCOL_005 <> 0) 

( s$$FCTCOL_006 <> 0) 
(s$$FCTCOL_007 <> 0) 

{ s $ $ FCTCOL_0 0 8 <> 0) 

( s $ $ FCTCOL_0 0 9 <> 0) 
( s $ $ FCTCOL_0 1 0 <> 0) 
{ s $ $ FCTCOL_0 1 1 <> 0) 
( s $ $ FCTCOL_0 1 2 <> 0) 
( s $ $ FCTCOL_0 1 3 <> 0) 
( S $ $ FCTCOL_0 1 4 <> 0) 
<s$$FCTCOL_015 <> 0) 
( s $ $ FCTCOL_0 1 6 <> 0) 
{ S $ $ FCTCOL_0 1 7 <> 0) 
( S $ $ FCTCOL_0 1 8 <> 0) 
(s$$FCTCOL_019 <> 0) 
( S $ $ FCTCOL_02 0 <> 0) 
( s $ $ FCTCOL_0 2 1 <> 0) 
( s $ $ FCTCOL_0 2 2 <> 0) 
( s $ $ FCTCOL_0 2 3 <> 0) 
( s $ $ FCTCOL_0 2 4 <> 0) 


— #BLOCK END# MakelLM 
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— Insert BOOKS for deltas with different dim keys 


— When a dimension key changes then we can simply insert all the new facts with the 

— new dimension keys 

-- Note that seq = 1 here because this is the second transaction on this date for 
-- this order. 


— IRM: Insert Rebook More 

^******* ********** ******************************** 

— #BLOCK_BEGIN# MakeIRM 

$ $SELECT_INTO_BEGIN [ $$ FCTTBL [ ] _IRM] 

SELECT 

tiss iss, 
tss_key ss_key, 
tdate_key date_key, 
ttranstype_key transtype_key, 

1 seq 

t$$DIMKEYR_01 $$DIMKEYR_01 
t $ $ DIMKEYR_0 2 $$DIMKEYR_02 
t$$DIMKEYR_03 $$DIMKEYR_03 
t$$DIMKEYR_04 $$DIMKEYR_04 
t$$DIMKEYR_05 $$DIMKEYR_05 
t$$DIMKEYR_06 $$DIMKEYR_06 
t$$DIMKEYRj)7 $$DIMKEYR_07 
t$$DIMKEYR_08 $$DIMKEYR_08 
t$$DIMKEYR_09 $$DIMKEYR_09 
t$$DIMKEYR_10 $$DIMKEYR_10 
t $ $ DEGKEY_0 1 $ $ DEGKEY_0 1 
t$$DEGKEY_02 $$DEGKEY_02 
t$$DEGKEY_03 $$DEGKEY_03 




t $ $ FCTCOL_0 0 1 
t $ $ FCTCOL_0 0 2 
t $ $ FCTCOL_0 0 3 
t $ $ FCTCOL_0 0 4 
t $ $ FCTCOL_0 0 5 
t$$FCTCOL_006 
t$$FCTCOL_007 
t $ $ FCTCOL_0 0 8 
t $ $ FCTCOL_0 0 9 
t $ $ FCTCOL_0 1 0 
t $ $ FCTCOL_0 1 1 
t $ $ FCTCOL_0 1 2 
t $ $ FCTCOL__0 1 3 
t $ $ FCTCOL_0 1 4 
t $ $ FCTCOL_0 1 5 
t $ $ FCTCOL_0 1 6 
t $ $ FCTCOL_0 1 7 
t$$FCTCOL_018 
t $ $ FCTCOL_0 1 9 
t $ $ FCTCOL_0 2 0 
t $ $ FCTCOL_0 2 1 
t$$FCTCOL_022 
t$$FCTCOL_023 
t$$FCTCOL 024 


$ $ FCTCOL_0 0 1 
$ $ FCTCOL_0 0 2 
$$FCTCOL_003 
$ $ FCTCOL_0 0 4 
$ $ FCTCOL_0 0 5 
$ $ FCTCOL_0 0 6 
$ $ FCTCOL_0 07 
$$FCTCOL_008 
$ $ FCTCOL_0 0 9 
$ $ FCTCOL_0 1 0 
$ $ FCTCOL_0 1 1 
$ $ FCTCOL_0 1 2 
$$FCTCOL_013 
$ $ FCTCOL_0 1 4 
$$FCTCOL_015 
$$FCTCOL_016 
$ $ FCTCOL_0 17 
$$FCTCOL_018 
$$FCTCOL_019 
$ $ FCTCOL_0 2 0 
$$FCTCOL_021 
$$FCTCOL_022 
$$FCTCOL_023 
$$FCTCOL 024 


$ $ SELECT_I NTO_BODY [ $ $ FCTTBL [ ] _I RM ] 
FROM 

$ $ FCTTBL [ ] _N FD d 

WHERE 

( 


OR 

OR 


( S$$DIMKEYR_06 <> t$$DIMKEYR_06) 
{s$$DIMKEYR_05 <> t$$DIMKEYR__05) 
(S$$DIMKEYR_07 <> t $ $ DIMKEYR_07 ) OR 
(s$$DIMKEYR_04 <> t$$DIMKEYR_04 ) OR 
(s$$DIMKEYR 08 <> t$$DIMKEYR 08) 


OR 
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( s$$DIMKEYR_03 <> t$$DIMKEYR_03 ) OR 
(S$$DIMKEYR__09 <> t$$DIMKEYR_09 ) OR 
( s$$DIMKEYR_02 <> t$$DIMKEYR_02 ) OR 
{S$$DIMKEYR_10 <> t$$DIMKEYR_10) OR 
(S$$DIMKEYR_01 <> t $ $ DIMKEYR_0 1 ) 

) 


AND 


OR 

OR 

OR 

OR 

OR 

OR 

OR 

OR 

OR 

OR 

OR 

OR 

OR 

OR 

OR 

OR 

OR 

OR 

OR 

OR 

OR 

OR 

OR 


( 

( t$ $ FCTCOL_0 0 1 
( t$ $ FCTCOL_0 0 2 
( t$ $ FCTCOL_0 0 3 
{ t $ $ FCTCOL__0 0 4 
( t $ $ FCTCOL_0 0 5 
( t $ $ FCTCOL_0 0 6 
( t $ $ FCTCOL_0 0 7 
( t $ $ FCTCOL_0 0 8 
( t$$FCTCOL_009 
{ t $ $ FCTCOL_0 1 0 
( t $ $ FCTCOL_0 1 1 
( t $ $ FCTCOL_0 1 2 
{ t $ $ FCTCOL_0 1 3 
( t $ $ FCTCOL_0 1 4 
(t$$FCTCOL_015 
( t $ $ FCTCOL_0 1 6 
(t$$FCTCOL__017 
( t $ $ FCTCOL_0 1 8 
( t $ $ FCTCOL_0 1 9 
( t $ $ FCTCOL_0 2 0 
(t$$FCTCOL_021 
(t$$FCTCOL_022 
(t$$FCTCOL_023 
(t$$FCTCOL_024 
) 


0) 
0) 
0) 
0) 
0) 
0) 
0 ) 
0 ) 
0 ) 
0 ) 
<> 0 ) 
<> 0 ) 
<> 

<> 

<> 

<> 

<> 

<> 

<> 

<> 

<> 

<> 

<> 

<> 


0) 

0) 

0) 

0) 

0) 

0 ) 

0 ) 

0 ) 

0 ) 

0 ) 

0 ) 

0 ) 


— # BLOCK END# MakeIRM 




^********************************************* 

/ ***®i******************it** ********************************************* ***********/ 

— #BLOCK__BEGIN# DropOutput 
$$DDL BEGIN 

$$DROP_TABLE_IF_EXISTS [$$FCTTBL[] $$NEXT] 

$ $ DROP _T ABLE_I F_EX I STS [ $ $ FCTTBL [ ] _I NC ] 

$$DDL_END 

— #BLOCK_END# DropOutput 

/***************************♦*********♦*********************************************/ 
— Create FC table in case force_close was 

/I***^*i **.. ******** ,***************************************************************/ 

- - # BLOC K_BEG IN# MakeFC 

DECLARE $$ VAR [f Coexists] $$EPIINT$$EOS 

$ $DDL_BEGIN_NO_DECLARE 

$ $ VAR_AS S I GN_B EG I N [ f C_e x i s t S ] 

SELECT COUNT (1) 

$$VAR_ASSIGN_INTO[ fc_exists] 

FROM $$SQLSERVER [ sysobj ects ] $$ORACLE [tabs ] 

WHERE 

$$ SQLSERVER! id = ob j ect__id ( 1 dbo . $ $ FCTTBL [ ] _FC ’ ) AND sysstat & Oxf = 3] 

$$ORACLE [ table_najne = UPPER ('$$ FCTTBL []_FC’ ) ] 

$ $ VAR_AS S I GN_END 

$$IF[$$VAR[fc_exists] = 0] 

$$DDL EXEC [ - 
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$ $ SELECT_INTO_BEGIN [ $ $ FCTTBL [ ] _FC ] 

SELECT 

* 

$ $ SELECT_I NTO_BODY [ $ $ FCTTBL [ ] _FC ] 

FROM 

$$FCTTBL[] S$CURR 

WHERE 

1=0 

1 

$$END_IF 

$$DDL_END 

— #BLOCK_END# MakeFC 

/****** ********************************************************** *******************/ 
— Create the incremental table ^ ^ , 

/* *********************************************************** ***********************/ 

— #BLOCK_BEGIN# MakeINC 

$$SELECT_INTO_BEGIN [$$ FCTTBL [ ] _INC] 

SELECT 


$$SELECT_INTO 
FROM $$ FCTTBL 
SELECT * FROM 
SELECT * FROM 
SELECT * FROM 
SELECT * FROM 
SELECT * FROM 
SELECT * FROM 
SELECT * FROM 
SELECT * FROM 


_BODY [ $ $ FCTTBL [ J _INC ] 
[]jriN UNION ALL 
$ $ FCTTBL [ ] _I L UNION ALL 
$$ FCTTBL [ ] _IR UNION ALL 
$$ FCTTBL []_IRD UNION ALL 
$$FCTTBL [ ] _IND UNION ALL 
$$ FCTTBL []_IRM UNION ALL 
$$ FCTTBL [ ] _ILM UNION ALL 
$$ FCTTBL []_FC UNION ALL 
$$ FCTTBL []_I DM 


— #BLOCK_END# MakeINC 

^****** ********************************************************** *******************/ 

— CR158: We want to load _IMI table and still keep the non-descending 

— order so that the clustered index on a fact table can be created 

— without sorting. This way can speed up significantly in creating a 

— clustered index on a very large already sorted fact table. 

y* *********************** ********************************************************** / 

— #BLOCK_BEGIN# MakelMI 

$$SELECT_INTO_BEGIN [$$ FCTTBL [ ] _IMI] 

SELECT 

$ $ SELECT_INTO_BODY [ $ $ FCTTBL [ ] _IMI ] 

FROM $ $ FCTTBL [ ] $ $ CURR 

WHERE datejcey >= (SELECT MIN (date_key) FROM $ $ FCTTBL [ ] _INC ) 

UNION ALL 

SELECT * FROM $$ FCTTBL [] _INC 
$$ SQLSERVER [ORDER BY 
date_key 
$$DIMKEYR_01 
$$DIMKEYR_02 
$ $ DIMKEYR_0 3 
$$DIMKEYR_04 
$ $ DIMKEYR_0 5 
$ $ DIMKEYR_0 6 
, $$DIMKEYR_07 

, $ $ DIMKEYR_0 8 

, $$DIMKEYR_09 

, $$DIMKEYR_10 

] 

— #BLOCK_END# MakelMI 

/***********************************************************************************/ 
— Create the new fact table and incremental table 
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— Note that transaction tables must be built before 

— these statements are run 

/ 


* * 


— #BLOCK_BEGIN# MakeNewFact 

$ $SELECT_INTO_BEGIN [ $ S FCTTBL [ ] $ $ NEXT ] 

SELECT * 

$$SELECT_INTO_BODY[$$FCTTBL[ ] $$NEXT] 

FROM $$FCTTBL[]$$CURR s » 

WHERE s.date_key < (SELECT MIN (date_key) FROM $$ FCTTBL [ ]_INC) 

UNION ALL 

SELECT * FROM $$ FCTTBL [] __IMI 
— #BLOCK_END# MakeNewFact 

/**,***»**. ************************************************************************* / 

^”H^L**.*.***.*«.**.**********************«*********v 

— #BLOCK_BEGIN# SPResults 

DECLARE $$VAR[count__INC] $$EPIINT$$EOS 

BEGIN 

$$VAR_ASSIGN_BEGIN [count__INC] 

SELECT COUNT (1) 

$ $ VAR_ASS IGN_INTO [ count_INC ] 

FROM $$ FCTTBL []_INC 
$ $ VAR_AS S I GN_EN D 

INSERT INTO adaptive_template_prof ile (token_name, number_rows) 

SELECT 'PROCESSED*, COUNT (1) FROM $$ FCTTBL [] _MFL$$ EOS 

INSERT INTO adaptive template prof ile (token name, number_rows) 

SELECT 'INSERTED', $$VAR [countllNC] - COUNT(l) FROM $$ FCTTBL [ ]_TIN$$EOS 

END$$EOS 

— #BLOCK_END# SPResults 

/************************************★*****************************/ 

— Set ioin order for SQL Server 

z**************************************************************** / 

— #BLOCK_BEGIN# ForcePlanOff 
$$ SQLSERVER [SET FORCEPLAN OFF] 


— # BLOCK END# ForcePlanOff 




— Drop temp tables and TXN and TIN 1:a ^ > ^ e ^^^ # ^^^^^ it ^^^^^^ + *^^*********^*************/ 


--#BLOCK BEGIN# DropTempsAf ter 


$$DDL_BEGIN 
S$DROP_TABLE_ 
$ $ DROP_T ABLE” 

$$drop_table“ 

$$drop_table“ 

$$DROP_TABLE_ 

$$drop_table“ 
$$drop_table’ 
$$drop_table_ 
$ $ DROP_TABLE_ 
$$drop_table" 
$$DROP TABLE" 


IF_EXISTS 
IF_EXISTS 
IF_EXISTS 
IF_EXISTS 
IF_EXISTS 
IF_EXISTS 
IF_EXISTS 
IF_EXISTS 
IF_EXISTS 
IF_EXISTS 
IF EXISTS 


[$$ FCTTBL [ ] _TIN] 
[$$ FCTTBL []_TMI] 
[ $ $ FCTTBL [ ] _FC ] 

[ $ $ FCTTBL [ ] __TXN ] 
[Concat_MFL] 

[ $ $ FCTTBL [ ] _1ST ] 
[$$ FCTTBL []_IL] 
[$$ FCTTBL []_IR] 
[$$ FCTTBL [ ] _IRD] 
[$$FCTTBL[]_IND] 
[ $$FCTTBL [ ] NFD] 
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$ $ DROP__TABLE_I F_EXI STS [ $ $ FCTTBL [ ] _I RM ] 
$ $ DROP_TABLE_I F_EXI STS [ $ $ FCTTBL [ ] _I DM ] 
$ $ DROP_TABLE_I F_EXI STS [ $ $ FCTTBL [ ] _I LM ] 
$ $ DROP_TABLE_I F_EX I STS [ $ $ FCTTBL [ ] _IMI ] 
$$DDL_END 

— #BLOCK_END# DropTempsAfter 

-#TEMPLATE_END# load_state 
— #TEMPLATE_BEGIN# load_trans 


/* 




Copyright * 1997, Epiphany Marketing Software, Inc. All Rights Reserved. 

— load_trans 

-- Move transaction-like staging data into Fact table - create a temp 
table with TXN extension that has all old rows along with new rows. 

Also produce a TIN (TXN INC) table that has only the new rows 

— Note that the new table will also include all existing rows from 

— the Fact table. 

/*★************************************************************************** 

/**** ****************************************************************** ****** 

— Delete output tables 

— output table is called TXN and includes old and new rows 

— Also, leave around _TIN as incremental table from this 
-- procedure 

— we also create a table called _TMI which contains all the 

— TIN records plus the records of overlapping period from the 

— old existing fact table. ^ , 

************************************************************************* 

— #BLOCK_BEGIN# RemoveOutput 

$$DDL_BEGIN 

$ $ DROP _TABLE_I F_EXI STS [ $ $ FCTTBL [ ] _TXN ] 

$ $ DRO P_T A B L E_ I F_EXI STS [ $ $ FCTTBL [ ] _TMI ] 

$ $ DROP_TABLE_I F_EXI STS [ $ $ FCTTBL [ ] _TI N ] 

$$DDL_END 

— #BLOCK_END# RemoveOutput 

/* ********************* *****+**************************************/ 

— Set join order for SQL Server 

/******;************************ ******************** ***************/ 

— #BLOCK_BEGIN# ForcePlanOn 
$$ SQLSERVER [SET FORCEPLAN ON] 

— #BLOCK_END# ForcePlanOn 

/**************★****************★******** ********************************** 

— Remove stuff already in fact table 

-- Note that currently this filter implies that once a transactional 

— fact entry is made it cannot be changed - and no further fact 

— entries on that date or any previous date can be made either 
/*********************************** ********************** ***************** 

— #BLOCK_BEGIN# CreateTIN 

$$SELECT__INTO_BEGIN [ $$ FCTTBL [ ] _TIN] 

SELECT 


*• * * * * i 
* * * * * i 


****** ; 
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S .1SS, 

s .ss_key, 
s .date_key, 
s . transtype_key, 
s.ikey seq 

, s . $ $ DIMKEYR_0 1 

, s.$$DIMKEYR_02 

, S . $ $ DIMKEYR_03 

, S . $ $ DIMKEYR_0 4 

, S.$$DIMKEYR_05 

, s.$$DIMKEYR_06 

f s . $$DIMKEYR_07 

, s.$$DIMKEYR_08 

, s . $ $ DIMKEYR__0 9 

, S .$$DIMKEYR_10 

, s . $$DEGKEY_01 

, S . $$DEGKEY_02 

, s . $ $ DEGKEY_0 3 

, s . $ $ FCTCOL_0 0 1 

, s.$$FCTCOL_002 

, s . $ $ FCTCOL_0 0 3 

, s . $ $ FCTCOL_0 0 4 

, s . $ $ FCTCOL_0 0 5 

, s . $ $ FCTCOL_0 0 6 

, s.$$FCTCOL_007 

, S . $ $ FCTCOL_008 

, s . $ $ FCTCOL_0 0 9 

, s . $ $ FCTCOL_0 1 0 

, s . $ $ FCTCOL_0 1 1 

, s . $ $ FCTCOL_0 1 2 

, s . $ $ FCTCOL_0 1 3 

, s . $ $ FCTCOL_0 1 4 

, s . $$FCTCOL_015 

, S . $ $ FCTCOL_0 1 6 

, s .$$FCTCOL__017 

, s . $$FCTCOL_018 

, S . $ $ FCTCOL_0 1 9 

, s . $ $ FCTCOL_0 2 0 

, s . $ $ FCTCOL_02 1 

, S.$$FCTCOL_022 

, s . $$FCTCOL_023 

, s . $ $ FCTCOL_0 2 4 

$ $ SELECT_I NTO_BODY [ $ $ FCTTBL [ ] _T IN ] 

FROM 

$ $ FSTGTBL [ ] _MAP s, bus_process b 

WHERE 

NOT EXISTS (SELECT * FROM $$ FCTTBL [] $$CURR f WHERE 
s.iss = f.iss AND 
s.ss_key = f.ss_key AND 
f.date_key >= s.date_key) 

AND ( 

( s . $ $ FCTCOL_0 0 1 <> 0) 

OR ( s . $ $ FCTCOL_002 <> 0) 

OR ( s . $ $ FCTCOL_0 0 3 <> 0) 

OR (s . $$FCTCOL_004 <> 0) 

OR ( s . $ $ FCTCOL_005 <> 0) 

OR (s.$$FCTCOL_006 <> 0) 

OR (s . $$FCTCOL_007 <> 0) 

OR (s.$$FCTCOL_008 <> 0) 

OR ( s . $ $ FCTCOL_0 0 9 <> 0) 

OR ( s . $$FCTCOL_010 <> 0) 

OR ( s . $ $ FCTCOL_0 1 1 <> 0) 

OR ( s . $ $ FCTCOL_0 1 2 <> 0) 

OR (s.$$FCTCOL_013 <> 0) 

OR (s.$$FCTCOL_014 <> 0) 

OR ( s . $ $ FCTCOL_0 1 5 <> 0) 

OR ( s . $ $ FCTCOL_0 1 6 <> 0) 

OR ( s . $ $ FCTCOL_0 1 7 <> 0) 

OR (s . $$FCTCOL 018 <> 0) 
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OR 

OR 

OR 

OR 

OR 

OR 

AND 


( s . $ $ FCTCOL_0 1 9 <> OT 
( s . $ $ FCTCOL_0 2 0 <> 0) 

( s . $ $ FCTCOL_0 2 1 <> 0) 

( s . $ $ FCTCOL_0 2 2 <> 0) 
(s.$$FCTCOL_023 <> 0) 
{s . $$FCTCOL_024 <> 0) 

) 


s . process key = b.process_key AND b.process_name 


= 'LoadTrans' 


— # BLOCK END# CreateTIN 


/ *************************** ****************** *********************/ 

— Set ioin order for SQL Server 

/******************************************************************/ 

-#BLOCK_BEGIN# ForcePlanOff 
$$ SQLSERVER [SET FORCEPLAN OFF] 

-#BLOCK_END# ForcePlanOff 

/*.***..*.****.*********************■ *******************************t************** / 

— CR158: We want to load _TMI table and still keep the non-descending 

— order so that the clustered index on a fact table can be created 

— without sorting. This way can speed up significantly in creating a 



— #BLOCK_BEGIN# CreateTMI 

$ $ SELECT_INTO_BEGIN [ $ $ FCTTBL [ ] _TMI ] 

SELECT 

* 

$ $ SELECT_INTO_BODY [ $ $ FCTTBL [ ] _TMI 3 
FROM 

$$ FCTTBL [ ] $$CURR 

WHERE date ke y >= ( SELECT MAX ( date__key ) FROM $$ FCTTBL [] _TIN) 

UNION ALL ” 

SELECT 

* 

FROM 

$$ FCTTBL [ ] _TIN 
$$ SQLSERVER [ORDER BY 
date_key 
$$DIMKEYR_01 
$$DIMKEYR_02 
$$DIMKEYR_03 
$$DIMKEYR_04 
$$DIMKEYR_05 
$ $ DIMKEYR_0 6 
$$DIMKEYR_07 
$$DIMKEYR_08 
$ $ DIMKEYR_0 9 
$$DIMKEYR_10 


-#BLOCK_END# CreateTMI 


*•**★*****************/ 


— #BLOCK_BEGIN# CreateTXN 

$ $ SELECT_INTO_BEGIN [ $ $ FCTTBL [ ] _TXN ] 

SELECT 

* 

$ $ SELECT_INTO_BODY [ $ $ FCTTBL [ ] _TXN ] 

FROM 
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$$FCTTBL [ ] $$CURR s 
WHERE s.datejcey < (SELECT MAX (date_key) FROM $$FCTTBL [ ] _TIN) 

UNION ALL 
SELECT 


FROM 


$ $ FCTTBL [ ] __TMI f 


— #BLOCK_END# CreateTXN 

/********************************************************************************* ^ 

Count inserted data and put results into communication table . 

— #BLOCK_BEGIN# SPResults 
BEGIN 

INSERT INTO adaptive_template__prof ile (token__name, number__rows) 

SELECT 1 PROCESSED’ , COUNT (1) FROM $ $ FSTGT BL [ ] _MAP $ $ EOS 

INSERT INTO adapt ive_template_prof ile (token_name, number_rows) 

SELECT ' INSERTED * , COUNT (1) FROM $$ FCTTBL [] _TIN$$EOS 

END$$EOS 

— #BLOCK_END# SPResults 
— #TEMPLATE_END# load_trans 
— #TEMPLATE_BEGIN# index_fact 

/ *,*********************************************************************************** / 

Copyright * 1997, Epiphany Marketing Software, Inc. All Rights Reserved. 

— Post processing after an extraction run 

— Reindex fact tables , . . , 

CR158: added WITH SORTED_DATA in creating cluster index on tact taoxe 


-- Remove any temp tables generated during the extraction 










**************************************************** 

;7*^™f?*^.i™«**-*******^« * 

— #BLOCK_BEGIN# PKIndexFact 

$$DDL_BEGIN 
$ $ DDL_EXEC [ 

CREATE UNIQUE INDEX XPK$ $ FCTTBL [ ] $ $ NEXT ON $ $ FCTTBL [ ] $ $NEXT 

( iss , ss_key , datejcey , transtypejcey , seq 

) 

] 

$$DDL_END 

— #BLOCK_END# PKIndexFact 

/*******************♦********************************************************** 
— #BLOCK BEGIN# IEIndexFact 


Mr*-**** I 
<■******/ 




$$DDL_BEGIN 
$$DDL EXEC [ 
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CREATE $$ SQLSERVER [CLUSTERED ] INDEX XIEK$$FCTTBL [ ] $$NEXT ON $$FCTTBL [ ] $$NEXT 

( 

date_key 

, $$DIMKEYR_01 

f $$DIMKEYR_02 

, $$DIMKEYR_03 

, $$DIMKEYR_04 

, $$DIMKEYR_05 

, $$DIMKEYR_06 

, $$DIMKEYR_07 

, $$DIMKEYR_08 

, $$DIMKEYR__09 

, $$DIMKEYR_10 

) $$ SQLSERVER [WITH SORTED_DATA] 


$$DDL_END 

— #BLOCK END# IEIndexFact 




— Remove any mapped tables ^ ***************************************/ 

— #BLOCK BEGIN# RemoveTemps 


$$DDL_BEGIN 

$ $ DROP_TABLE_I F_EXI STS [ $ $ FSTGTBL [ ] _MAP ] 
$$DDL_END 

— #BLOCK_END# RemoveTemps 


— #TEMPLATE_END# index_fact 
-“#TEMPLATE_BEGIN# ren_trans 


Copyright * 1997, Epiphany Marketing Software, Inc. All Rights Reserved. 


ren_trans 

Epiphany Marketing Software, 1997 


Simply change the name of the transaction new table to the 
actual fact table name - used for Fact tables that don't have 
any stored procedure other than load_trans attached to them 


t* 






— Delete the output tables ^ *********★★**★★*★★★**★**/ 


— #BLOCK_BEGIN# RemoveOutput 


$$DDL_BEGIN 

$$DROP_TABLE_IF_EXISTS [$$FCTTBL [ ] $$NEXT] 

$ $ DROP_TABLE_I F_EXI STS [ $ $ FCTTBL [ ] _INC j 
$$DDL_END 

— #BLOCK_END# RemoveOutput 

^*********** *.***.****•*■*★★★*★*★****•*****★*•*■**★****************************/ 

— Move all transaction rows into the correct new fact table 

— name. Note that we would use sp_rename, except it 

— doesn’t work with DB name prefixes 

— TBD : Rename instead of re-select 

★****************************************************/ 
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—# BLOCK BEGIN# BuildNewFact 


$ $ SELECT_INTO_BEGIN [ $ $ FCTTBL [ ] $ $NEXT ] 

SELECT 

* 

$$SELECT_INTO_BODY [$$FCTTBL [ ] $$NEXT] 

FROM 

$ $ FCTTBL [ ] _TXN 
— #BLOCK_END# BuildNewFact 

y******************************** ******** *******************************/ 

— Preserve incremental table ^ , 

/********************* ******** *************************** ***************/ 


— #BLOCK_BEGIN# Buildlncremental 

$$SELECT_INTO_BEGIN [ $$FCTTBL [ ] _INC] 
SELECT 

* 

$ $SELECT_INTO_BODY [$$ FCTTBL [ ] _INC] 
FROM 

$$FCTTBL[]_TIN 

— #BLOCK END# Buildlncremental 


/ 

/ 






********* 


- Count inserted data and put results into communication table 
*********************************************************************** 


/ 

/ 


— #BLOCK_BEGIN# SPResults 
BEGIN 


INSERT INTO adapt ive__template_prof ile {token_name, number__rows) 
SELECT 'PROCESSED', COUNT(l) FROM $$ FCTTBL [] _TXN$$ EOS 

INSERT INTO adaptive_template_prof ile (token_name, numbe r__rows ) 
SELECT 'INSERTED', COUNT (1) FROM $$ FCTTBL [] _TXN$$ EOS 

END$$EOS 

— # BLOCK END# SPResults 






— Remove temp tables 

y**************************************************** 




/ 

/ 


— #BLOCK_BEGIN# RemoveTemps 
$$DDL_BEGIN 

$ $ DROP_TABLE_I F_EXI STS [ $ $ FCTTBL [ ] _TXN ] 

$ $ D RO P_T AB L E_ I F_EXI STS [ $ $ FCTTBL [ ] _TIN] 

$ $ DRO P_T A B L E_ I F_EXISTS [ $ $ FCTTBL [ ] _TMI ] 

$$DDL_END 

— #BLOCK_END# RemoveTemps 
— #TEMPLATE_END# ren_trans 
— #TEMPLATE_BEGIN# map_keys 

y*************************************** ************** *************/ 

— Copyright * 1997, Epiphany Marketing Software, Inc. All Rights Reserved. 

— map_keys 

— Epiphany Marketing Software 
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— Map dimension keys from Staging table and report 

— on unjoined rows 

^***************************★**************************************7 

^****************** ************★★**********************************/ 

— Remove output table 

y^*****************************************************************/ 

— #BLOCK_BEGIN# DropTemp 
$$DDL_BEGIN 

$$DROP_TABLE_IF_EXISTS [$$FSTGTBL [ ]_MAP] 

$$DDL_END 

— #BLOCK_END# DropTemp 

/jr + irl' + ii'ieitic-kiHr + ie************************************************"**"**/ 

— Set join order for SQL Server 

y******************************************************************/ 

— #BLOCK_BEGIN# ForcePlanOn 
$$ SQLSERVER [SET FORCE PLAN ON] 

— #BLOCK_END# ForcePlanOn 

^******************************************************************/ 

— Map dimension keys via Inner joins 

/★•a****************************************************************/ 

— #BLOCK_BEGIN# MapAll 

$ $SELECT_INTO_BEGIN [ $ $ FSTGTBL [ ] _MAP] 

SELECT 

s.iss, 
s . ss_key, 
s . date_key, 
s .transtype_key, 
s .ikey, 
s ,process_key 
, $$PIPE STATE 


m_04 
m_03 
m_06 
m_02 
m_08 
m_05 
m_09 
m_01 
m_07 
m 10 


$$DIMKEY_04 
$$DIMKEY_03 
$$DIMKEY_06 
, $$DIMKEY__02 
,$$DIMKEY_08 
. $$DIMKEY_05 
, $$DIMKEY_09 
,$$DIMKEY_01 
, $$DIMKEY_07 
, $$DIMKEY 10 


$ $ DIMKEYR_0 4 
$$DIMKEYR_03 
$ $ DIMKE YR_0 6 
$ $ DIMKEYR_02 
$$DIMKEYR_08 
$ $ DIMKEYR_0 5 
$$DIMKEYR_09 
$$DIMKEYR_01 
$$DIMKEYR_07 
$$DIMKEYR 10 


$$DEGKEY_03 

$$DEGKEY_02 

$$DEGKEY_01 

S . $$FCTCOL_001 
S.$$FCTCOL_002 
S.$$FCTCOL_003 
s . $$FCTCOL__004 
s . $ $ FCTCOL_0 0 5 
s . $ $ FCTCOL__0 0 6 

S.$$FCTCOL_007 
s . $ $ FCTCOL_00 8 
s . $ $ FCTCOL_00 9 
s . $ $ FCTCOL_0 1 0 
S . $ $ FCTCOL_0 1 1 
S.$$FCTCOL_012 
S . $$FCTCOL 013 
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s . $ $ FCTCOL_0 1 4 
s . $ $ FCTCOL_0 1 5 
s . $ $ FCTCOL_0 1 6 
S . $ $ FCTCOL_0 1 7 
s.$$FCTC0Lj)18 
s . $ $ FCTCOL_0 1 9 
S . $ $ FCTCOL_0 2 0 
S . $$FCTCOL_021 
S .$$FCTCOL_022 
S . $$FCTCOL_023 
s . $$FCTCOL 024 


$ $ SELECT_XNTO_BODY [ $ $ FSTGTBL [ ] 
FROM 

$$FSTGTBL [ ] S 

, $ $MAPTBL_0 4 $ $NEXT m_04 

, $ $MAPTBL_0 3$ $NEXT m_03 

, $ $MAPTBL_0 6$ $NEXT m_06 

, $ $ MAPTBL_0 2 $ $NEXT m_02 

, $$MAPTBL_08$$NEXT m_08 

, $$MAPTBL_05$$NEXT m_05 

, $ $MAPTBL_0 9 $ $NEXT m_09 

, $ $MAPTBL_0 1 $ $ NEXT m_01 

, $$MAPTBL_07$$NEXT m_07 

, $$MAPTBL 10$$NEXT m 10 


$$SQLSERVER[ {index = 1) 
$$ SQLSERVER [ (index = 1) 
$$SQLSERVER[ (index = 1) 
$$ SQLSERVER [ (index = 1) 
$$ SQLSERVER ( (index = 1) 
$$SQLSERVER[ (index = 1) 
$$SQLSERVER[ (index = 1) 
$$ SQLSERVER [ (index = 1) 
$$SQLSERVER[ (index = 1) 
$$SQLSERVER[ (index = 1) 


WHERE 1=1 


s.iss AND 
s.iss AND 
s.iss AND 
s.iss AND 
s.iss AND 
s.iss AND 
s.iss AND 
s.iss AND 
s.iss AND 
s.iss AND 


. $ $ DSTGKEY_0 4 
. $ $DSTGKEY_03 
. $ $ DSTGKEY_0 6 
. $$DSTGKEY_02 
. $$DSTGKEY_08 
. $$DSTGKEY_05 
. $$DSTGKEY_09 
. $$DSTGKEY_01 
. $$DSTGKEY_07 
. $$DSTGKEY 10 


S . $ $ DSTGKEYR_0 4 
S.$$DSTGKEYR_03 
s . $ $ DSTGKE YR_0 6 
s.$$DSTGKEYR_02 
S.$$DSTGKEYR_08 
S.$$DSTGKEYR_05 
s . $$DSTGKEYR__09 
s.$$DSTGKEYR_01 
s . $$DSTGKEYR_07 
s . $$DSTGKEYR 10 


— # BLOCK_EN D # MapAll 

^********************************* ★★*★★***★**★***•********★★* *★***★*/ 

— Set join order for SQL Server 

/******************-**************^*********************************/ 
— #BLOCK_BEGIN# ForcePlanOff 
$$ SQLSERVER [SET FORCEPLAN OFF] 

— #BLOCK_END# ForcePlanOff 

y************************ *****★**★★*★***★★*★ ******************★****/ 

— Look for unjoined data. Report on processed rows 
/*********★********************************************************/ 

— #BLOCK_BEGIN# SPResults 

$$DECLARE_BEGIN 

$$DECLARE_BODY[$$VAR[un joined] $$EPIINT] 

$ $ DECLARE_BODY [$$VAR[ processed] $$EPIINT] 


$$VAR_ASSIGN_BEGIN[ processed] 
SELECT COUNT (1) 

$ $ VAR_AS S I GN_I NTO [processed] 

FROM $$FSTGTBL [ ] 

$ $ VAR_AS S I GN_EN D 

$$ VAR_AS S I GN_BEGIN[ unjoined] 

SELECT $$VAR [processed] - COUNT (1) 
$$VAR ASSIGN INTO [unjoined] 
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FROM $$FSTGTBL [ ] _MAP 
$$VAR_ASSIGN_END 

INSERT INTO adaptive_template_profile (token_name, number_rows) 

SELECT 'UNJOINED', $$VAR [unj oined] $$NO_FROM_LIST$$EOS 

INSERT INTO adapt ive_template_profile (token_name, number_rows) 

SELECT 'PROCESSED', $$VAR [processed] $$NO_FROM_LIST$$EOS 

INSERT INTO adaptive_template_prof ile (token_name, numbe r_rows ) 

SELECT 'INSERTED', $$VAR [processed] - $$VAR [unjoined] $$NO_FROM_LIST$$EOS 

END$$EOS 

— #BLOCK_END# SPResults 

/★★it***************************************************************/ 

— Index this temp table 

/****★******★**★*********★*******★*********************************/ 

— #BLOCK_BEGIN# IndexMap 

$$DDL_BEGIN 

$$DDL_EXEC[ 

CREATE INDEX X$ $ FSTGTBL [ ] _MAP ON $ $ FSTGTBL [ ] _MAP 

( 

iss, ss_key, date_key, ikey 

) 

] 

$$DDL_END 

— #BLOCK_END# IndexMap 

— #TEMPLATE_END# map_keys 
— #TEMPLATE_BEGIN# upd_unj 

/***********************★*********************************************/ 

— Copyright * 1997, Epiphany Marketing Software, Inc. All Rights Reserved 

— upd_unj 

— Epiphany Marketing Software 

— Update all dimension keys to 'UNKNOWN' in staging table 

— where referential integrity fails 

/******************★★*************************************************/ 

/***★★★**★******★******★*********************•*************************/ 

— Count the number of rows to update in the staging table - that is, those 

— that have at least one Foreign key where referential integrity fails 
/*******************★*************************************************/ 

--#BLOCK_BEGIN# CountUnj 

BEGIN 

INSERT INTO adaptive_template_profile { token jiame, number_rows) 

SELECT 'PROCESSED', COUNT(l) FROM $$ FSTGTBL []$$ EOS 

INSERT INTO adaptive_template_prof ile (token_name, number_rows) 

SELECT 'MODIFIED', COUNT (1) 

FROM 

$$ FSTGTBL [ ] s 
WHERE 1=0 

OR NOT EXISTS (SELECT 1 FROM $$MAPTBL_04$$NEXT m_04 WHERE m_04.iss = s.iss AND 
m_04.$$DSTGKEY_04 = $ $ DSTGKE YR_0 4 ) 

OR NOT EXISTS (SELECT 1 FROM $$MAPTBL_03$$NEXT m_03 WHERE m_03.iss = s.iss AND 
m_0 3 . $ $ DSTGKE Y_0 3 = $$DSTGKEYR_03 ) 

OR NOT EXISTS (SELECT 1 FROM $$MAPTBL 06$$NEXT m 06 WHERE m 06.iss = s.iss AND 
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m_06.$$DSTGKEY_06 = $ $ DSTGKEYR_0 6 ) 

OR NOT EXISTS (SELECT 1 FROM $$MAPTBL_02$$NEXT m_02 WHERE m_02.iss 
m_02 . $$DSTGKEY_02 = $$ DSTGKE YR_0 2 ) 

OR NOT EXISTS (SELECT 1 FROM $$MAPTBL_08$$NEXT m_08 WHERE m_08.iss 
m_08.$$DSTGKEY_08 = $ $ DSTGKE YR__0 8 ) 

OR NOT EXISTS (SELECT 1 FROM $$MAPTBL_05$$NEXT m_05 WHERE mJDS.iss 
m_05.$$DSTGKEY_05 = $ $ DSTGKE YR_0 5 ) 

OR NOT EXISTS (SELECT 1 FROM $$MAPTBL_09$$NEXT m_09 WHERE rn_09.iss 
m_09.$SDSTGKEY_09 = $ $ DSTGKEYR__0 9 ) 

OR NOT EXISTS (SELECT 1 FROM $$MAPTBL_01$$NEXT m_01 WHERE m_01.iss 
m_01.$$ DSTGKE Y_01 = $ $ DSTGKEYR_0 1 ) 

OR NOT EXISTS (SELECT 1 FROM $$MAPTBL_07$ $NEXT m_07 WHERE m_07.iss 
m_07.$$DSTGKEY_07 = $$DSTGKEYR_07 ) 

OR NOT EXISTS (SELECT 1 FROM $$MAPTBL_10$$NEXT m_10 WHERE m_10.iss 
m_10.$$DSTGKEY_10 = $$DSTGKEYR_10) 

$$EOS 
END$$EOS 

— #BLOCK_END# CountUnj 

/****************★******************★*********************************/ 

-- Update foreign keys where referential integrity fails 

/★★fr******************************************************************/ 

— #BLOCK_BEGIN# UpdateUnj $$DSTGKEYR_04 

UPDATE $$FSTGTBL [ ] SET $$ DSTGKE YR_0 4 = 'UNKNOWN' 

WHERE NOT EXISTS (SELECT 1 FROM $$MAPTBL_04$$NEXT m 

WHERE m.iss = $$FSTGTBL [ ] . iss AND m. $$DSTGKEY_04 = $$FSTGTBL [ ] . $$DSTGKEYR_04 ) 

--#BLOCK_END# UpdateUnj $$ DSTGKEYRJ) 4 

— #BLOCK_BEGIN# UpdateUnj $$DSTGKEYR_03 

UPDATE $$FSTGTBL[] SET $ $ DSTGKEYR_0 3 = 'UNKNOWN' 

WHERE NOT EXISTS (SELECT 1 FROM $$MAPTBL_03$$NEXT m 

WHERE m.iss = $$FSTGTBL [ ] . iss AND m. $$ DSTGKE Y_0 3 = $$FSTGTBL [ ] . $$DSTGKEYR_03 ) 

— # BLOCK_END# UpdateUnj $ $ DSTGKEYR_0 3 

— #BLOCK_BEGIN# UpdateUnj $ $ DSTGKEYR_0 6 

UPDATE $$FSTGTBL [ ] SET $ $ DSTGKEYR_0 6 = 'UNKNOWN' 

WHERE NOT EXISTS (SELECT 1 FROM $$MAPTBL_06$$NEXT m 

WHERE m.iss = $$FSTGTBL [ ] .iss AND m. $$DSTGKEY_06 = $$FSTGTBL [] . $$DSTGKEYR_06) 

— # BLOCK_END# UpdateUnj $ $ DSTGKEYR_0 6 

— #BLOCK_BEGIN# UpdateUnj $$DSTGKEYR_02 

UPDATE $$FSTGTBL [ ] SET $ $ DSTGKE YR_0 2 = 'UNKNOWN' 

WHERE NOT EXISTS (SELECT 1 FROM $$MAPTBL_02$$NEXT m 

WHERE m.iss = $$FSTGTBL [ ] . iss AND m. $$ DSTGKE Y_0 2 = $$FSTGTBL[] . $$DSTGKEYR_02 ) 

— # BLOC K_EN D # UpdateUnj $$DSTGKEYR_02 

— #BLOCK_BEGIN# UpdateUnj $$ DSTGKE YR_0 8 

UPDATE $$FSTGTBL [ ] SET $ $ DSTGKE YR_0 8 « 'UNKNOWN' 

WHERE NOT EXISTS (SELECT 1 FROM $$MAPTBL_08$$NEXT m 

WHERE m.iss = $$FSTGTBL[] .iss AND m. $$DSTGKEY_08 = $$FSTGTBL [ ] . $$DSTGKEYR_08 ) 

— #BLOCK_END# UpdateUnj $$DSTGKEYR_08 

— #BLOCK_BEGIN# UpdateUnj $$DSTGKEYR_05 

UPDATE $$FSTGTBL[] SET $$DSTGKEYR_05 = 'UNKNOWN' 

WHERE NOT EXISTS (SELECT 1 FROM $$MAPTBL_05$$NEXT m 

WHERE m.iss = $$FSTGTBL [ ] . iss AND m. $$ DSTGKE Y_0 5 = $$FSTGTBL[] . $$DSTGKEYR_05 ) 

— # BLOCK END# UpdateUnj $$DSTGKEYR 05 
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= S.iss AND 
= S.ISS AND 
= s.iss AND 
= s.iss AND 
= s.iss AND 
= s.iss AND 
= s.iss AND 


— #BLOCK_BEGIN# UpdateUnj $$DSTGKEYR_0 9 


UPDATE $$FSTGTBL [ ] SET $$DSTGKEYR_09 = 'UNKNOWN 1 

WHERE NOT EXISTS (SELECT 1 FROM $$MAPTBL_09$$NEXT m 

WHERE m.iss = $$FSTGTBL [ ] . iss AND m. $$DSTGKEY_09 = $$FSTGTBL [ ] 


. $$DSTGKEYR_09) 


— #BLOCK_END# UpdateUnj $$DSTGKEYR_09 


— #BLOCK_BEGIN# UpdateUnj $$DSTGKEYR_01 


UPDATE $$FSTGTBL [ ] SET $$DSTGKEYR__01 = 'UNKNOWN' 

WHERE NOT EXISTS (SELECT 1 FROM $$MAPTBL_01$$NEXT m 

WHERE m.iss = $$FSTGTBL [ ] . iss AND m. $$DSTGKEY_01 = $$FSTGTBL[] 


. $$DSTGKEYR_01 ) 


— #BLOCK_END# UpdateUnj $ $DSTGKEYR_01 


— #BLOCK_BEGIN# UpdateUnj $$DSTGKEYR_07 


UPDATE $$FSTGTBL[] SET $$DSTGKEYR_07 = 'UNKNOWN' 
WHERE NOT EXISTS (SELECT 1 FROM $$MAPTBL_07$$NEXT m 
WHERE m.iss = $$FSTGTBL[] .iss AND m. $$DSTGKEY_07 = 


$$FSTGTBL [ ] . $ $ DSTGKEYR_0 7 ) 


— #BLOCK_END# UpdateUnj $ $DSTGKEYR_07 


— #BLOCK_BEGIN# UpdateUnj $ $ DSTGKEYR_1 0 


UPDATE $$FSTGTBL [ ] SET $ $ DSTGKEYR_1 0 = 'UNKNOWN' 

WHERE NOT EXISTS (SELECT 1 FROM $$MAPTBL_10$$NEXT m 

WHERE m.iss = $$FSTGTBL [ ] . iss AND m. $$DSTGKEY_10 = $$FSTGTBL [ ] 


. $$DSTGKEYR_10) 


— #BLOCK_END# UpdateUnj $ $ DSTGKEYR_1 0 


— #TEMPLATE_END# upd_unj 


The following are the post-parsed SQL source for the adaptive templates as filled in with 


corresponding schema definitions. 


/ 


— #TEMPLATE_BEGI N # force_close 




— Copyright * 1997, Epiphany Marketing Software, Inc. All Rights Reserved. 

— force_close 

— Close out deleted orders - those that no longer appear in the 

— staging table 

— SEE SAFETY VALVE BELOW 

/*******************************★**************************************************/ 


/********************** 

— Delete temporary tables 
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— #BLOCK_BEGIN# DropTemps 

IF EXISTS (SELECT 1 FROM sysobjects WHERE id = object_id ( ' dbo . Order_0_FC 1 ) AND sysstat & Oxf 
3) DROP TABLE Order_0_FC 


— #BLOCK_END# DropTemps 

/★★★★★★★★A********************************** 

— Insert negative BOOKs for deleted orders 




— FC: ForceClose 

y* ******************************* *** 
— #BLOCK BEGIN# MakeFC 




SELECT 


f .iss, 
f .ss_key, 

MAX ( f ,date_key) date_key, 

MIN ( f . transtype_key) transtype_key , 
MAX ( f . seq) + 1 seq 
f . customerbillto_key 
f .product_key 
f . application_key 
f .program_key 
f . customershipto_key 
f . territory_key 
f .warehouse_key 

-SUM ( f . net_price ) net_price 
-SUM ( f . number_units ) number_units 


INTO Order_0_FC 
FROM 

Order_0_A f 

WHERE 

NOT EXISTS 

(SELECT 1 FROM Orders tage_MAP s WHERE s.iss = f.iss AND s.ssjcey = f. 

GROUP BY 

f .iss, 
f . ss_key 

, f . customerbxllto_key 

, f .product_key 

, f . application_key 

, f . program_key 

, f .customershipto_key 

, f . temtory_key 

, f ,warehouse__key 


ss_key) 


(SUM ( f . net_price) <> 0) 
(SUM { f . nuinber_units) <> 0) 


MIN ( f ,transtype_key) <= 99 

AND 

MIN ( f . transtype_key ) >= 1 
— #BLOCK_END# MakeFC 

^* *************************************************************************** 

— SAFETY VALVE - THIS PROC ONLY DOES ANYTHING 

— IF THE STAGING TABLE HAS AT LEAST ONE ROW 

y************************************************************************* + *^ 
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— #BLOCK_BEGIN# SafetyValue 
DECLARE @count MAP INT 


SELECT @count_MAP = { 
SELECT COUNT (1) 

FROM Orders tage_MAP 


IF {(@count_MAP = 0)) 
DELETE FROM Order 0 FC 


-#BLOCK_END# SafetyValue 


— Count processed, inserted rows 






— #BLOCK BEGIN# SPResults 


INSERT INTO adaptive_template_prof ile (token_name, number_rows) 
SELECT 'PROCESSED', COUNT (1) FROM Order_0_A 

INSERT INTO adaptive_template_prof ile (token_name, number_rows) 
SELECT 'INSERTED', COUNT (1) FROM Order_0_FC 


-#BLOCK END# SPResults 


— # TEMPLATE END# force close 


— # TEMPLATE BEGIN# load state 






— Copyright * 1997, Epiphany Marketing Software, Inc. All Rights Reserved. 


— load_state 

— Load order bookings into fact table by creating transactional 

— data from state data 

— load trans must be run before this procedure to create TIN table 








— Delete temporary tables ^ ***********y 


— #BLOCK_BEGIN# DropTemps 

IF EXISTS (SELECT 1 FROM sysobjects WHERE id = obj ect_id { ' dbo . Order_0_MFL ' ) AND sysstat & Oxf 
= 3) DROP TABLE Order_0_MFL 

IF EXISTS (SELECT 1 FROM sysobjects WHERE id = obj ect_id ( ’ dbo . Order_0_lST ' ) AND sysstat & Oxf 
= 3) DROP TABLE Order_0_lST 

IF EXISTS (SELECT 1 FROM sysobjects WHERE id = object_id { ' dbo . Order_0_IL ' ) AND sysstat & Oxf = 
3) DROP TABLE Order_0_IL 

IF EXISTS (SELECT 1 FROM sysobjects WHERE id = object_id { ' dbo . Order_0_IR’ ) AND sysstat & Oxf = 
3} DROP TABLE Order 0 IR 
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IF EXISTS 
= 3) DROP 

(SELECT I FROM sysobjects 
TABLE Order 0 IRD 

WHERE 

id 

- 

object_id ( ' dbo.Order_0_IRD ' ) 

AND 

sysstat 

& 

Oxf 

IF EXISTS 
= 3) DROP 

(SELECT 1 FROM sysobjects 
TABLE Order 0 IND 

WHERE 

id 


ob j ect_id ( ' dbo . Order_0_IND ' ) 

AND 

sysstat 

& 

Oxf 

IF EXISTS 
= 3) DROP 

(SELECT 1 FROM sysobjects 
TABLE Order 0 NFD 

WHERE 

id 

— 

ob ject_id ( ' dbo .Order_0_NFD ' ) 

AND 

sysstat 

& 

Oxf 

IF EXISTS 
= 3) DROP 

(SELECT 1 FROM sysobjects 
TABLE Order_0_IRM 

WHERE 

id 

— 

object id ( ’ dbo .Order_0_IRM' ) 

AND 

sysstat 

& 

Oxf 

IF EXISTS 
= 3) DROP 

(SELECT 1 FROM sysobjects 
TABLE Order 0 I DM 

WHERE 

id 

= 

object_id( 'dbo.Order_0_IDM' ) 

AND 

sysstat 

& 

Oxf 

IF EXISTS 
= 3) DROP 

(SELECT 1 FROM sysobjects 
TABLE order 0 ILM 

WHERE 

id 

— 

object id ( , dbo.Order_0_ILM* ) 

AND 

sysstat 

& 

Oxf 

IF EXISTS 
- 3) DROP 

(SELECT 1 FROM sysobjects 
TABLE Order 0 IMI 

WHERE 

id 

“ 

ob j ect_id { ' dbo . Order_0_IMI 1 ) 

AND 

sysstat 

& 

Oxf 


— #BLOCK_END# DropTemps 

/**********★*****************★*★*★*****•*■**★**★*********************/ 

— Set join order for SQL Server 

/*******★***★******★★*★**★*********★★*★★*****★★******************★*/ 

— #BLOCK_BEGIN# ForcePlanOn 
SET FORCEPLAN ON 

— # BLOCK_END# ForcePlanOn 

/****★★********★*********+*************************★*★******************************/ 

— Remove rows older than fact table - history can not be rewritten - only 

— the last date for an order can be changed. Note that we compare transtype's 

— because SHIP type transactions might occur at a later date and we don't want 

— those to interfere 

— Also, since the staging table may have multiple entries for a given order on 

— a single day - we assume that the list one inserted m the Staging table will 

— be used {since ikey is an IDENTITY column) 

— Note that a given ss_key must use the same Booking transtype for all of time, 

— otherwise the transtype_key 

— MFL : Mapped Filtered 

/************★******★*******★**********★ ************** *************** ***************/ 
— #BLOCK BEGIN# MakeMFL 


SELECT 

s . * 

INTO Order_0_MFL 
FROM 

OrderStage_MAP s, bus_process b 

WHERE 

( (s . date_key >= (SELECT MAX (datejcey) FROM OrderJ)_A f WHERE 
s.iss - f.iss AND s.ss_key = f.ss_key AND 
s . transtype_key = f . transtype_key) ) 

OR NOT EXISTS (SELECT * FROM Order_0_A f WHERE 

s.iss = f.iss AND s.ss_key = f.ss_key AND 
s . transtype_key = f . transtype_key) ) 

AND s . ikey = (SELECT MAX (t. ikey) FROM Orders tage_MAP t WHERE 
s.iss = t.iss AND 
s.ss_key = t.ss_key AND 

s. date_key = t.date_key AND 

t . process_key = b.process_key) 

AND 

s .process_key = b.process_key AND b .process_name = 'LoadState' 

— #BLOCK_END# MakeMFL 

/A**********************************************************************************/ 

— Index MFL table for later queries 
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--#BLOCK BEGIN# IndexMFL 


EXEC ( ’ 

CREATE INDEX XOrder_0_MFL ON Order_0_MFL 

( 

iss, ss_key, date_key 

) 

’) 


--#BLOCK_END# IndexMFL 

/***********************************★************************★**********************/ 

— Get oldest state rows for each unique sskey 

— We need to treat the first entry for each order 

— in the staging table separately from all others, since 

— only the first entry needs to be compared with 

— already existing fact entry rows to create transactions. 

— All subsequent dates for that order in the Fact table 

— can be delta ’d with other staging table entries - see the 

— section below on Pairwise deltas. 

— MFL should be indexed 

— 1ST: The first record for each iss, ss_key 
/***★******************★*********************★**★★***★*****************+************/ 

— #BLOCK BEGIN# MakelST 


SELECT 

s . * 

INTO Order_0_lST 
FROM 

Order_0_MFL s 

WHERE 

s . date_key = (SELECT MIN (date_key) FROM Order_0_MFL t WHERE 
s.iss = t.iss AND s.ss_key = t.ss_key) 

— #BLOCK_END# MakelST 

/*****************★*****************+************************************* **********/ 
— Index 1ST for later queries 

/******★**★*********★**********************★★********★******************************/ 
— #BLOCK BEGIN# IndexlST 


EXEC ( ’ 

CREATE UNIQUE INDEX XPKOrder_0_lST ON Order_0_lST 
( 

iss, ss_key 

) 


— #BLOCK__END# IndexlST 

/********************★***************★**********★**********★★★************★*********/ 

— Insert negative BOOKS for changed dim keys 

— This query will add up all existing Books and Loss’s 

— for this order and the net facts will be cancelled out 

— with the old Dimension keys. Note that an invariant of this 

— procedure is that only one set of dimensions at a time 

— can have non-zero facts. 
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— Fact table Should be indexed 

— HAVING Clause is needed to prevent changing of dimensions 

— on fully shipped order from causing a transaction - no sense 

— creating fact rows with all zero's in them 

— Note that we increment the sequence number just in case 

— this new transaction occurs on the same date as the last 

— existing one in the fact table - to avoid index errors 

— IL: insertLost 

/**★**★★**★*★*★***********************★*******+******★★**★**************************/ 
— # BLOCK BEGIN# MakelL 


SELECT 

s .iss, 
s . ss__key, 
s .date_key, 
s . transtype_key, 

MAX(f.seq) + 1 seq 
, f .customerbillto_key 

, f .product_key 

f f .application_key 

, f . program_key 

, f . customershipto_key 

, f . territory__key 

, f . warehouse__key 

, -SUM ( f . net_price ) netjorice 

, -SUM { f . number_units ) number_units 

INTO Order_0_IL 
FROM 

Order_0_lST s, Order_0_A f 

WHERE 

s.iss = f.iss AND s.ss_key = f.ss_key 

AND 

( {s . territory_key <> f . territory_key) OR 

(s ,customershipto_key <> f . customershipto_key) OR 

( s .warehouse_key <> f . warehouse_key) OR 

( s .program_key <> f .program_key) OR 

(s .application_key <> f . application_key) OR 

( s .product_key <> f .product_key) OR 

<s .customerbillto_key <> f . customerbillto_key) ) 

GROUP BY 

s . iss, 
s . ss_key, 
s .date_key, 
s . transtype_key 
, f ,customerbillto_key 

, f .product_key 

, f . application_key 

, f .program_key 

, f . customershipto_key 

, f . territory_key 

, f . warehouse_key 

HAVING 

MIN (f . transtype_key) = s . transtype_key 

AND 

< 

( SUM ( f .net_price) <> 0) 

OR (SUM(f ,number_units) <> 0) 

) 

— #BLOCK_END# MakelL 

/★**★***•*■**•***★*★★**★*★★■*■★**•★★*★★*★*★**•*■****★*******★*★•*■★•*■★***'*•*'******•*•**•*******•*•*•*••*'/ 
— Index IL for later queries 
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ji >y ii. 




—# BLOCK BEGIN# IndexIL 


EXEC ( 1 

CREATE INDEX XPKOrder_0_IL ON Order_0_IL 
( 

iss, ss_key 

) 

’) 


— #BLOCK_END# IndexIL 

/****★★**★*★★*★★*★★*★★**★*★***★**★***★•****★★*******★****■*■***★•*■★*•********■***★★*****★** i 

— Insert BOOKS for changed dim keys 

— When a dimension changes then just create a booking 

— transaction for whatever we negated above with the new 

— dimension and fact values 

— 1ST shoud be indexed 

— Note that we add one to whatever we used as the last 

— seq because this transaction occurs on the same 

— date as the negative one above 

— IR: Insert Rebook 
— #BLOCK BEGIN# MakeIR 


SELECT 

s . iss, 
s . ss_key, 
s .date_key, 

1 . transtype_key, 
l.seq + 1 seq 

, s . customerbillto_key 

, s .product_key 

, s .application_key 

, s .program_key 

, s . customershipto_key 

, s . territory_key 

, s . warehouse_key 

, -l.net_price net_price 

, -1 . number_units number^units 

INTO Order_0_IR 
FROM 

Order_0_IL 1, Order_0_lST s 
WHERE l.iss = s.iss AND l.ss_key = s.ss_key 

— # BLOCK_END# MakeIR 

— Insert BOOKs for changed dim keys where fact 

— also changed 

— When a dimension changes at the same time as 

— a fact then we need to make up the fact difference 

— 1ST shoud be indexed 

— Note that we add two to whatever we used as the last 

— seq because this transaction occurs on the same 

— date as the negative and positive ones above 
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— Note also that the Left Outer join uses transtype_key 

— so that only the Bookings at the old value will be counted. 

— Whereas above for the negative transaction value 

— we want to include Shipments in our calculation, here 

— we only want to see how Booking Facts have changed. 

— Here again, only one Booking transaction type is supported 

— per ss_key 

— IRD: Insert Rebook delta 

/***********************************************************************************/ 
— #BLOCK BEGIN# MakeIRD 


SELECT 

s . iss, 
s . ss_key, 
s .date_key, 
s . transtype_key, 
l.seq + 2 seq 

, s.customerbillto_key 

, s.product^key 

, s. application_key 

, s.program_key 

, s .customershipto_key 

, s . territory_key 

, s . warehouse_key 

, MAX (s ,net_price) -ISNULL (SUM { f ,net_price ) , 0) net_price 

, MAX (s.number_units) -ISNULL {SUM (f.number^units) , 0) number_umts 

INTO Order_0_IRD 
FROM 

Order_0_IL 1, Order_0_lST s 

LEFT OUTER JOIN Order_0_A f ON s.iss = f.iss AND s.ss_key = f.ss_key AND 
s. transtype_key = f . transtype_key 
WHERE 

l.iss = s.iss AND l.ss_key = s.ss_key 

GROUP BY 

s.iss, 
s . ss_key, 
s .date_key, 
s . transtype_key, 

1 . seq 

, s .customerbillto_key 

, s .product_key 

, s . application_key 

, s .program_key 

, s .customershipto_key 

, s . territory_key 

, s . warehouse_key 

HAVING 

(ISNULL {SUM (f .net_price) , 0) <> MAX{s.net_price) ) 

OR (ISNULL (SUM (f.number_units) , 0) <> MAX (s . number_units) ) 

— #BL0CK_END# MakeIRD 

/*************★***★******★*****★******************************★*********************/ 

— Insert BOOKS for deltas with same dim keys OR for 

— brand new orders. 

— Note that we DON’T want to count Shipments 

— (so shipment ss_key's should be different from 

— order ss_keys) since we just want bookings to sum up 

— to whatever this transcation says they should be. 

— Fact table should be indexed 
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— WHERE clause prevents double booking on changed 

— dimension - if we didn't use the NOT EXISTS clause 

— then this query would repeat the work of the last one 

— above - which we have already taken care of 

— HAVING clause ensures that multiple 0 records don't 

— get inserted whenever this procedure is run 

— Note that we increment the sequence number just in case 

— this new transaction occurs on the same date as the last 

— existing one in the fact table - to avoid index errors 

— IND: Insert New Delta 

y*********************************************************** 

--#BLOCK BEGIN# MakeIND 


s .iss, 
s . ss_key, 
s .date_key, 
s . transtype_key, 

ISNULL (MAX(f .seq) , 0) + 1 seq 

, s . customerbillto_key 

, s ,product_key 

, s . application_key 

, s ,program_key 

, s . customershipto_key 

, s . territory_key 

, s . warehouse_key 

, MAX {s .netjprice) -ISNULL (SUM ( f.net_pnce) , 0) net_price 

, MAX (s .number_umts) -ISNULL (SUM (f.number_units) , 0) number_units 

INTO Order_0_IND 
FROM 

Order_0_lST s LEFT OUTER JOIN Order_0_A f ON 

s.iss = f.iss AND s.ss_key = f.ss_key AND f . trans type_key = s . transtype_key 

WHERE 

NOT EXISTS (SELECT * FROM Order_0_IL WHERE iss = s.iss AND ss_key = s.ss_key) 


GROUP BY 


s . iss, 

s . ss_key, 

s.date_key, 

s.transtype_key 

s . customerbillto_key 

s.product_key 

s . application_key 

s.program_key 

s . customershipto_key 

s . terntory_key 

s . warehouse_key 


HAVING 

(ISNULL ( SUM ( f . net_price) , 0) <> MAX (s . net_price) ) 

OR (ISNULL ( SUM ( f . number_units) , 0) <> MAX (s .number_units) ) 

— #BLOCK_END# MakeIND 

/************★*******★************★*★★*★***************★**************★*************/ 

— Form pairwise deltas for all rows except earliest for each sskey 

— Each row created in NFD will consist of two sequential entries from the 

— staing table. So if N enties for an order exist in MFL (after we have filtered 

— out same-date duplicates) then all the queries above will deal with the earliest entry, 
whereas 

— all the queries below (including this one) will deal with the N-l deltaing transactions 
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— This query assumes that MFL will already have been filtered 

— to have a single record for each order/datekey 

— NFD: Not First Delta 

/★**★****★************-**★****★**★*****★*****★******★*****+***★**★***********+*******/ 
— #BLOCK BEGIN# MakeNFD 


SELECT 

s.iss siss, t.iss tiss 
, s.ss_key sss_key, t.ss_key tss_key 

, s.date_key sdate_key, t.date_key tdate_key 

, s . transtype_key stranstype_key, t. transtype_key ttranstype_key 

, s .customerbillto_key scustomerbillto_key, t . customerbillto_key tcustomerbillto_key 

, s .product_key sproduct_key, t ,product_key tproduct_key 

, s . application_key sapplication_key, t . application_key tapplication_key 

, s .program_key sprogram_key, t .program_key tprogram_key 

, s . customershipto_key scustomershipto_key, t . customershipto_key tcustomershipto_key 

, s . territory_key sterritory_key, t . territory_key tterritory_key 

, s .warehouse_key swarehouse_key, t . warehouse_key twarehouse_key 

, s.net_price snet_pnce, t.net_price tnet_price 

, s .number_umts snumber_umts, t ,number_units tnumber_units 

INTO Order_0_NFD 
FROM 

Order_0_MFL s, Order_0_MFL t 

WHERE 

s.iss = t.iss AND s.ss_key = t.ss_key 

AND 

s.date_key = {SELECT MAX (datejcey) FROM Order_0_MFL u WHERE 
u.iss = s.iss AND u.ss_key = s.ss_key AND u.date_key < t.date_key) 

— #BLOCK_END# MakeNFD 

/*******★***********★**★****★★******************************★*★★********************/ 

— Insert BOOKs for deltas with same dim keys 

— If the dimensions don’t change then we create a 

— new booking order (as long as at least one of the facts 

— have changed) 

— I DM: Insert Delta More 

^★★★★******************************************************'*******'***'**********'A'****/ 

— # BLOCK BEGIN# MakelDM 


SELECT 

tiss iss, 
tss_key ss_key, 
tdate_key date_key, 
ttranstype_key transtype_key, 

0 seq 

, tcustomerbillto_key customerbillto_key 

, tproduct_key product_key 

, tapplication_key application_key 

, tprogram_key program_key 

, tcustomershipto_key customershipto_key 

, tterritory_key territory_key 

, twarehouse_key warehouse_key 

, tnet_pnce-snet_price net_pnce 

, tnumber_units-snumber_units number_units 

INTO Order_0__I DM 
FROM 

Order_0_NFD d 

WHERE 
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( 

(sterritory_key = tterritory_key) AND 
(scustomershipto_key - tcustomershipto_key) AND 
(swarehouse_key = twarehouse_key) AND 
(sprogram_key = tprogram_key) AND 
(sapplication_key - tapplication_key) AND 
(sproduct_key = tproduct_key) AND 
(scustomerbiIlto_key = tcustomerbillto__key) 

) 

AND 

( 

(snet_price <> tnet_price) 

OR (snumber_units <> tnumber_units) 

) 

— #BLOCK_END# MakelDM 

/★****★★********★**★**★***★**★•*:*★***★★**★**★★* + *★**★*•**★★★*******★★*★★★*************/ 

— Insert negative BOOKS for deltas with different dim keys 

— If one of the dimensions change then we first create a lose transaction for 

— all the previous facts. (Negate all the facts from the earlier of the two 

— transactions) 

— ILM: Insert Lost More 

/***********************************★*******************★**************★*******★*★**/ 
--#BLOCK BEGIN# MakeILM 


SELECT 

si ss iss, 
sss_key ss_key, 
tdate_key date_key, 
stranstype_key transtype_key, 

0 seq 

, scustomerbillto_key customerbillto_key 

, sproduct_key product_key 

, sapplication_key application_key 

, sprogram_key program_key 

, scustomershipto_key customershipto_key 

, sterritory_key terrrtory_key 

, swarehouse_key warehouse_key 

, -snet_price net_price 

, -snumber_units number_units 

INTO Order_0_ILM 
FROM 

Order_0_NFD d 

WHERE 

( 

(sterritory_key <> tterritory_key) OR 
(scustomershipto_key <> tcustomershipto_key) OR 
(swarehouse_key <> twarehouse_key) OR 
(sprogram_key <> tprogram_key) OR 
(sapplication_key <> tapplication_key) OR 
(sproduct_key <> tproduct_key) OR 
(scustomerbillto_key <> tcustomerbillto_key) 

) 

AND 

( 

(snet_pnce <> 0) 

OR (snumber_units <> 0) 

) 

— #BLOCK_END# MakeILM 

/***★*****★•*■**★•**★**★***★* + •*■•**★★******★**★**•*•***★**■***★★★★★★*★*★★★*★**•***•■*•*★★*★****■*'/ 
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— Insert BOOKs for deltas with different dim keys 

— When a dimension key changes then we can simply insert all the new facts with the 

— new dimension keys 

— Note that seq = 1 here because this is the second transaction on this date for 

— this order. 

— IRM: Insert Rebook More 

/***************************+*****+*■***■****■**■*******■***************'***********•*** ie-k-k j 

— # BLOCK BEGIN# MakeIRM 


SELECT 

tiss iss, 
tss_key ss_key, 
tdate_key date_key, 
ttranstype_key transtype_key, 

1 seq 

, tcustomerbillto_key customerbillto_key 

, tproduct_key product_key 

, tapplication_key application_key 

, tprogram_key program_key 

, tcustomershipto_key customershipto_key 

, tterritory_key territory_key 

, twarehouse_key warehouse_key 

, tnet_price net_price 

, tnumber_units number_units 

INTO Order_0_IRM 
FROM 

Order_0_NFD d 

WHERE 

( 

(sterritory_key <> tterritory_key) OR 
(scustomershipto_key <> tcustomershipto_key) OR 
(swarehouse_key <> twarehouse_key) OR 
(sprogram_key <> tprogram_key) OR 
(sapplication_key <> tapplication_key) OR 
{ sproduct_key <> tproduct_key) OR 
( scustomerbillto_key <> tcustomerbillto_key) 

) 

AND 

( 

(tnet_price <> 0) 

OR ( tnumber_units <> 0) 

) 

— #BLOCK_END# MakeIRM 
— Delete the output tables 
— #BLOCK_BEGIN# DropOutput 


IF EXISTS (SELECT 1 FROM sysobjects WHERE id = obj ect_id ( ' dbo. Order_0_B ' ) AND sysstat & Oxf = 
3) DROP TABLE Order_0_B 

IF EXISTS (SELECT 1 FROM sysobjects WHERE id = object_id ( 1 dbo. Order_0_INC ’ ) AND sysstat & Oxf 
= 3) DROP TABLE Order_0_INC 


- - # BLOCK_END# DropOutput 
— Create FC table in case force close was 
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— not run 

/***********************************************************+*********************** j 

— #BLOCK_BEGIN# MakeFC 
DECLARE @fc exists INT 


SELECT @fc_exists = ( 

SELECT COUNT (1) 

FROM sysobjects 
WHERE 

id = object_id ( 'dbo.Order_0_FC' ) AND sysstat & Oxf = 3 


IF (@fc_exists = 0) 
EXEC { 1 


INTO Order_0_FC 
FROM 

Order_0_A 

WHERE 

1=0 

') 


-#BLOCK END# MakeFC 




— Create the incremental table 


-# BLOCK BEGIN# MakeINC 


INTO Orde 
FROM Orde 
SELECT * 
SELECT * 
SELECT * 
SELECT * 
SELECT * 
SELECT * 
SELECT * 
SELECT * 


r_0_INC 
r_0_TIN UNION 
FROM Order_0_ 
FROM Order_0_ 
FROM Order_0_ 
FROM Order_0_ 
FROM Order_0_ 
FROM Order_0_ 
FROM Order_0_ 
FROM Order 0 


ALL 

IL UNION ALL 
IR UNION ALL 
IRD UNION ALL 
IND UNION ALL 
IRM UNION ALL 
ILM UNION ALL 
FC UNION ALL 
I DM 


-#BLOCK END# MakeINC 








— CR158: We want to load _IMI table and still keep the non-descending 

— order so that the clustered index on a fact table can be created 

— without sorting. This way can speed up significantly in creating a 

— clustered index on a very large already sorted fact table. 

y**********************************************^************************************y 

— #BLOCK BEGIN# MakelMI 


INTO Order_0_IMI 
FROM Order_0_A 

WHERE date key >= (SELECT MIN (date key) FROM Order 0 INC) 


Method and Apparatus for Creating a Well- PATENT 
Formed Database System Using a Computer Page 143 


Attorney Docket No 20308 702 
ODMA\PCDOCS\SQL ] \2280 17U 


Inventors: Craig D. Weissman, 
Greg V. Walsh and Eliot L. Wegbreit 




UNION ALL 

SELECT * FROM Order_0_INC 
ORDER BY 

date_key 

, customerbillto_key 

, product_key 

, application_key 

, program_key 

, customershipto_key 

, territory_key 

, warehouse_key 


— #BLOCK_END# MakelMI 

y***********************************************************************************/ 

— Create the new fact table and incremental table 

— Note that transaction tables must be built before 

— these statements are run 

y**********************************+************************************************/ 
— #BLOCK BEGIN# MakeNewFact 


SELECT * 

INTO Order__0_3 
FROM Order_0_A s 

WHERE s.date_key < {SELECT MIN {date_key) FROM Order_0_INC) 

UNION ALL 

SELECT * FROM Order_0_IMI 
— #BLOCK_END# MakeNewFact 

/★**********************************************************************************/ 

— Count processed, inserted rows 

/*******************★★*******★*****★******★*****★***★*******************************/ 
— #BLOCK_BEGIN# SPResults 
DECLARE @COunt_INC INT 
BEGIN 

SELECT @count_INC = ( 

SELECT COUNT (1) 

FROM Order_G_INC 

) 

INSERT INTO adaptive_template_prof ile (token_name, number_rows) 

SELECT ’PROCESSED’, COUNT (1) FROM Order_0_MFL 

INSERT INTO adapt ive_template_prof ile (token_name, number_rows) 

SELECT ’INSERTED’, @count_INC - C0UNT{1) FROM Order_0_TIN 

END 

— #BLOCK_END# SPResults 

— Set join order for SQL Server 

y***************************-***-**********'*** , ***’A'*******************/ 

— #BLOCK_BEGIN# ForcePlanOff 
SET FORCEPLAN OFF 
— #BLOCK_END# ForcePlanOff 

y***********************************************************************************y 
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— Drop temp tables and TXN and TIN table 

y-*************************-*******-**************:*-** **★★**★*★★***•***★*■***■**★★*****★* y 


— #BLOCK_BEGIN# DropTempsAf ter 


IF EXISTS (SELECT 1 FROM sysobjects WHERE id 
= 3) DROP TABLE Order_0_TIN 

IF EXISTS (SELECT 1 FROM sysobjects WHERE id 
= 3) DROP TABLE Order_0_TMI 

IF EXISTS (SELECT 1 FROM sysobjects WHERE id 
3) DROP TABLE Order_0_FC 

IF EXISTS (SELECT 1 FROM sysobjects WHERE id 
= 3) DROP TABLE Order_0_TXN 

IF EXISTS (SELECT 1 FROM sysobjects WHERE id 
3) DROP TABLE Concat_MFL 

IF EXISTS (SELECT 1 FROM sysobjects WHERE id 
= 3) DROP TABLE Order_0_lST 

IF EXISTS (SELECT 1 FROM sysobjects WHERE id 
3) DROP TABLE Order_0_IL 

IF EXISTS (SELECT 1 FROM sysobjects WHERE id 
3) DROP TABLE Order_0_IR 

IF EXISTS (SELECT 1 FROM sysobjects WHERE id 
= 3) DROP TABLE Order_0_IRD 

IF EXISTS (SELECT 1 FROM sysobjects WHERE id 
= 3) DROP TABLE Order_0_IND 

IF EXISTS (SELECT 1 FROM sysobjects WHERE id 
= 3) DROP TABLE Order_0_NFD 

IF EXISTS (SELECT 1 FROM sysobjects WHERE id 
- 3) DROP TABLE Order_0_IRM 

IF EXISTS (SELECT 1 FROM sysobjects WHERE id 
= 3) DROP TABLE Order _0_IDM 

IF EXISTS (SELECT 1 FROM sysobjects WHERE id 
= 3) DROP TABLE Order_0_ILM 

IF EXISTS (SELECT 1 FROM sysobjects WHERE id 
= 3) DROP TABLE Order_0_IMI 


obj ect_id { 1 dbo . Order_0 _TIN ' ) 
ob j ect_id ( ' dbo . Order_0_TMI ' ) 
ob j ect_id ( 1 dbo . Order_0_FC ’ ) 
ob j ect_id ( ' dbo . Order_0_TXN ' ) 
ob j ect_id ( ' dbo . Concat_MFL ' ) 
ob j ect_id ( 1 dbo . Order_0_lST ' ) 
ob j ect_id ( ' dbo . Order_0_IL ’ ) 
object_id ( , dbo.Order_0_IR’ ) 
ob j ect_id ( ' dbo . Order_0_IRD ' ) 
obj ect_id ( ' dbo . Order_0_IND 1 ) 
ob j ect_id ( ' dbo . Order_0_NFD 1 ) 
object_id ( ’ dbo.Order_0__IRM' ) 
object_id ( ' dbo.Order_0_IDM’ ) 
object_id ( ’ dbo.Order_0_ILM' ) 
object_id ( ' dbo. Order_0_IMI ' ) 


AND sysstat & Oxf 
AND sysstat & Oxf 
AND sysstat & Oxf = 
AND sysstat & Oxf 
AND sysstat & Oxf = 
AND sysstat & Oxf 
AND sysstat & Oxf “ 
AND sysstat & Oxf = 
AND sysstat & Oxf 
AND sysstat & Oxf 
AND sysstat & Oxf 
AND sysstat & Oxf 
AND sysstat & Oxf 
AND sysstat & Oxf 
AND sysstat & Oxf 


— #BLOCK__END# DropTempsAf ter 

— #TEMPLATE_END# load_state 
— #TEMPLATE_BEGIN# load_trans 

/★a*******************************************************************************/ 


— Copyright * 1997, Epiphany Marketing Software, Inc. All Rights Reserved. 

— load_trans 

— Move transaction-like staging data into Fact table - create a temp 

— table with TXN extension that has all old rows along with new rows. 

— Also produce a TIN (TXN INC) table that has only the new rows 

— Note that the new table will also include all existing rows from 

— the Fact table. 

y************************^********************************************************y 


/**★★**★*★*★★*★****★*★***★★******★***★**★*****★★**★***★*********★★*★★★**★★*******★/ 

— Delete output tables 

— Output table is called TXN and includes old and new rows 

— Also, leave around _TIN as incremental table from this 

— procedure 

— We also create a table called _TMI which contains all the 

— _TIN records plus the records of overlapping period from the 

— old existing fact table. 
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— #BLOCK_BEGIN# RemoveOutput 


IF EXISTS (SELECT 1 FROM sysobjects WHERE id = ob ject_id { ' dbo . Order_0_TXN ' ) AND sysstat & Oxf 
= 3) DROP TABLE Order_0_TXN 

IF EXISTS (SELECT 1 FROM sysobjects WHERE id = objected (' dbo. Order_0_TMI ' ) AND sysstat & Oxf 
= 3) DROP TABLE Order_0_TMI 

IF EXISTS (SELECT 1 FROM sysobjects WHERE id = object_id { 'dbo.Orde^OjriN* ) AND sysstat & Oxf 
= 3) DROP TABLE Order_0_TIN 

— #BLOCK__END# RemoveOutput 


— Set join order for SQL Server 






— #BLOCK_BEGIN# ForcePlanOn 
SET FORCEPLAN ON 

— #BLOCK_END# ForcePlanOn 

/***********★*★★★********★****************★***********★***********★**★**********★*/ 

— Remove stuff already in fact table 

— Note that currently this filter implies that once a transactional 

— fact entry is made it cannot be changed - and no further fact 

— entries on that date or any previous date can be made either 

— # BLOCK BEGIN# CreateTIN 


S . 1SS, 

s . ss_Jcey, 
s .date_key, 
s . transtype_key, 
s.ikey seq 

, s . customerbillto_key 

, s .product_key 

, s . application_key 

, s .program_Jcey 

, s.customershipto_key 

, s . temtory_key 

, s .warehouse_key 

, s.net_price 

, s . number_units 

INTO Order_0_TIN 
FROM 

Orders tage_MAP s, bus_process b 

WHERE 

NOT EXISTS (SELECT * FROM Order_0_A f WHERE 
s.iss = f.iss AND 
s.ss_key = f.ss_key AND 
f.date_key >= s.date_key) 

AND ( 

(s.net_price <> 0) 

OR (s. number units <> 0) 


s.process_key = b ,process_key AND b.process_name = 'LoadTrans' 
BLOCK END# CreateTIN 




— Set join order for SQL Server 
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— #BLOCK_BEGIN# ForcePlanOff 
SET FORCEPLAN OFF 
— #BLOCK_END# ForcePlanOff 

^*********************************************************************************/ 

— CR158 : We want to load __TMI table and still keep the non-descending 

— order so that the clustered index on a fact table can be created 

— without sorting. This way can speed up significantly in creating a 

— clustered index on a very large already sorted fact table. 
/********★**********★★*★*★★************★***********★******************■*•******★****/ 

— #BLOCK BEGIN# CreateTMI 


SELECT 

* 

INTO Order_0_TMI 
FROM 

Order_0_A 

WHERE 

date_key >= (SELECT MAX (date_key) FROM Order_0_TIN) 
UNION ALL 
SELECT 

* 

FROM 

Order_0_TIN 
ORDER BY 

date_key 

, customerbillto_key 

, product_key 

, application_key 

, program_key 

, customershipto_key 

, territory_key 

, warehouse_key 


— #BLOCK_END# CreateTMI 

/***********★****************★************★*********★***********************★★****/ 

— Insert everything into the new fact table 
/★****★**********************★****★*******★**********★****************************/ 

— #BLOCK BEGIN# CreateTXN 


SELECT 

★ 

INTO Order_0_TXN 
FROM 

Order_0_A s 

WHERE s.datejcey < (SELECT MAX (date_key) FROM Order_0_TIN) 

UNION ALL 
SELECT 

* 

FROM 

Order_0_TMI f 
— #BLOCK_END# CreateTXN 

/★*******************************★**★***************★*****************************/ 
— Count inserted data and put results into communication table 
/***********★*****************★******★★*******★*********★****★********★***★*******/ 

— #BLOCK_BEGIN# SPResults 

BEGIN 
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INSERT INTO adaptive_template_prof lie (tokerwiame, number_rows) 

SELECT ' PROCESSED 1 , COUNT (1) FROM OrderStage _MAP 

INSERT INTO adaptive__templatej?rof ile (token_name, number_rows) 

SELECT ' INSERTED 1 , COUNT(l) FROM Order_0_TIN 

END 

— #BLOCK_END# SPResults 
— #TEMPLATE_END# load_trans 
— #TEMPLATE_BEGIN# index_fact 

/********★***************★*★****★****★*★★**********★**********************************/ 

— Copyright * 1997, Epiphany Marketing Software, Inc. All Rights Reserved. 

— Post processing after an extraction run 

— Reindex fact tables 

— CR158 : added WITH SORTED_DATA in creating cluster index on fact table 

— Remove any temp tables generated during the extraction 
/*****★**************★**★****★**★*★*★**★*****★★***************************************/ 

/********************+**************************+*********************************+***/ 

— Primary key index the fact table 

/★★a**********************************************************************************/ 

— #BLOCK BEGIN# PKIndexFact 


EXEC ( ' 

CREATE UNIQUE INDEX XPKOrder_0_B ON Order_0_B 
( 

iss , ss_key , date_key , transtype_key , seq 

) 

') 


— # BLOC K_EN D # PKIndexFact 

/★★****** ***************************************************************** ************/ 

— Inversion index the fact table 

/**+************************★************★******★*************************************/ 
— #BLOCK BEGIN# IEIndexFact 


EXEC ( ' 

CREATE CLUSTERED INDEX XIEKOrder_0_B ON Order_0_B 
( 

date_key 

, customerbillto_key 

, product_key 

, application_key 

, program_key 

, customershipto_key 

, territory_key 

, warehouse_key 

) WITH SORTED_DATA 

*) 


— #BLOCK END# IEIndexFact 
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-- Remove any mapped tables ^ ****/ 

— #BLOCK_BEGIN# RemoveTemps 


IF EXISTS {SELECT 1 FROM sysobjects WHERE id = object_id ( ' dbo. Order Stage_MAP ' ) AND sysstat & 
Oxf = 3) DROP TABLE Order St age_MAP 


— #BLOCK_END# RemoveTemps 

— #TEMPLATE_END# index_fact 
— #TEMPLATE_BEGIN# ren_trans 

/****★****★*************************************************************/ 

— Copyright * 1997, Epiphany Marketing Software, Inc. All Rights Reserved. 

— ren_trans 

— Epiphany Marketing Software, 1997 

— Simply change the name of the transaction new table to the 

— actual fact table name - used for Fact tables that don’t have 

— any stored procedure other than load_trans attached to them 

^************** ****************+***********★********** + **★******★*** ****/ 

/*********★*************************************************************/ 

— Delete the output tables 

^* ***************** ******* ******★*********************★*****************/ 

— #BLOCK_BEGIN# RemoveOutput 

IF EXISTS (SELECT 1 FROM sysobjects WHERE id = obj ect_id { ' dbo . Order_0_B ' ) AND sysstat & Oxf = 
3) DROP TABLE Order_0_B 

IF EXISTS {SELECT 1 FROM sysobjects WHERE id = object_id { ' dbo . Order_0_INC ' ) AND sysstat & Oxf 
= 3) DROP TABLE Order_0_INC 


— #BLOCK_END# RemoveOutput 

/*********★****★***+********★*******************************************/ 

— Move all transaction rows into the correct new fact table 

— name. Note that we would use sp_rename, except it 

— doesn't work with DB name prefixes 

— TBD: Rename instead of re-select 

/**************************************+********************************/ 
--#BLOCK BEGIN# BuildNewFact 


SELECT 

* 

INTO Order_0_B 
FROM 

Order_0_TXN 

— #BLOCK_END# BuildNewFact 

/*******★***************************************************************/ 
— Preserve incremental table 

/***********************************************************************/ 
— #BLOCK_BEGIN# Buildlncremental 
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SELECT 


★ 

INTO Order_0_INC 
FROM 

Order_0_TIN 

— #BLOCK_END# Buildlncremental 

/*************★*********************************************************/ 

— Count inserted data and put results into communication table 

/****************★*************** *******★*********★*************★*★*****/ 

— #BLOCK_BEGIN# SPResults 

BEGIN 

INSERT INTO adaptive_template_prof ile <token_name, number_rows) 

SELECT 'PROCESSED 1 , COUNT (1) FROM Order_0_TXN 

INSERT INTO adaptive_template_prof ile (token_name, number_rows) 

SELECT ' INSERTED’ , COUNT (1) FROM Order_0_TXN 

END 

--#BLOCK_END# SPResults 

/***★****★***********★*******************************•*******************/ 

— Remove temp tables 

/********★*★******★**********•**★****************************************/ 
— #BLOCK_BEGIN# RemoveTemps 


IF EXISTS (SELECT 1 FROM sysobjects WHERE id 
= 3} DROP TABLE Order_0_TXN 

IF EXISTS (SELECT 1 FROM sysobjects WHERE id 
= 3) DROP TABLE Order_OjTIN 

IF EXISTS (SELECT 1 FROM sysobjects WHERE id 
= 3) DROP TABLE Order_0_TMI 


ob j ect_id ( 1 dbo . Order__0_TXN 1 ) AND sysstat & Oxf 
object_id{ * dbo.Orde^OjTIN 1 ) AND sysstat & Oxf 
object_id{'dbo.Order_0_TMI’) AND sysstat & Oxf 


— #BLOCK_END# RemoveTemps 
— #TEMPLATE_END# ren_trans 
— #TEMPLATE_BEGIN# map_keys 

/*****★***★***★***★********★*********★*•******★***********★*********/ 

— Copyright * 1997, Epiphany Marketing Software, Inc. All Rights Reserved. 

— map_keys 

— Epiphany Marketing Software 

— Map dimension keys from Staging table and report 

— on unjoined rows 

/****★*************************+***★★******************************/ 

/******************************************************************/ 

— Remove output table 

/*★***★*****★****************★*******★★★***************************/ 

— #BLOCK_BEGIN# DropTemp 

IF EXISTS (SELECT 1 FROM sysobjects WHERE id = object_id { ’ dbo . OrderStage_MAP 1 ) AND sysstat & 
Oxf = 3) DROP TABLE OrderStage_MAP 
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— #BLOCK_END# DropTemp 


/*****★************************************************************/ 
-- Set join order for SQL Server 

/************** ********** ****************** ************** **********/ 
— #BLOCK_BEGIN# ForcePlanOn 
SET FORCEPLAN ON 
— #BLOCK_END# ForcePlanOn 

/****************★*************************************************/ 
— Map dimension keys via Inner joins 
/******************************************************************/ 

— #BLOCK BEGIN# MapAll 


s.iss, 
s . ss_key, 
s .date_key, 
s . transtype_key, 
s . ikey, 
s .process_key 

m_04 . program_key program_key 

m_03 . application_key application_key 

m_06. territory_key territory_key 

m_02 * product_key product_key 

m_05 . customer_key customershipto_key 

m_0 1 . cus tome r_key customerbillto_key 

m 07 . warehouse_key warehouse_key 


, s.net_pnce 

, s .nurober_units 

INTO OrderStage_MAP 
FROM 

OrderStage s 

, ProgramMap_B m_04 (index = 1) 

, ApplicationMap_B m_03 (index = 1) 

, TerritoryMap_B m_06 (index = 1) 

, ProductMap_B m_02 (index = 1) 

, CustomerMap_B m_05 (index = 1) 

, CustomerMap_B m_01 (index = 1) 

, WarehouseMap_B m_07 (index = 1) 

WHERE 1=1 

AND m_04.iss = s.iss AND m_04 .program_sskey = s .prograra_sskey 

AND m~03.iss = s.iss AND m__03 .application_sskey = s .application_sskey 

AND m“06.iss = s.iss AND m__0 6. territory_ss key = s . territory_sskey 

AND m_02 . iss = s.iss AND m_02 .product_sskey = s ,product_sskey 

AND m__05 . iss = s.iss AND m_0 5 . customers s key = s .customershipto_sskey 

AND m_01 . iss = s.iss AND m_01 . customer_sskey = s .cus tome rbillto_ss key 

AND m_07 . iss = s.iss AND m__07 .ware house__ss key = s . warehouse_sskey 

— #BLOCK_END# MapAll 

y****** *************** ***************** *********************** *****/ 

— Set join order for SQL Server 

/*★★***************************************************************/ 

— #BLOCK_BEGIN# ForcePlanOff 
SET FORCEPLAN OFF 

— #BLOCK END# ForcePlanOff 
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— Look for unjoined data, Report on processed rows ^ 

--#BLOCK BEGIN# SPResults 


DECLARE @un joined INT 
DECLARE @processed INT 

BEGIN 

SELECT Gprocessed = ( 

SELECT COUNT (1) 

FROM OrderStage 
) 

SELECT @un}oined = ( 

SELECT @processed - COUNT (1) 

FROM OrderStage_MAP 
) 

INSERT INTO adaptive_template_prof lie (token_name, number_rows ) 
SELECT 'UNJOINED', Qunjoined 

INSERT INTO adaptive_template jprofile (token_name, number_rows) 
SELECT 'PROCESSED', @processed 

INSERT INTO adaptive_template_prof ile (token_name, number_rows) 
SELECT 'INSERTED', Gprocessed - Gun joined 

END 

— #BLOCK_END# SPResults 

^******************************************************************/ 
— Index this temp table 

**************************************** *************/ 
— #BLOCK_BEGIN# IndexMap 


EXEC { ' 

CREATE INDEX XOrderStage_MAP ON OrderStage_MAP 
( 

iss, ss_key, date_key, ikey 

) 

') 


— #BLOCK_END# IndexMap 

— #TEMPLATE_END# map_keys 
— #TEMPLATE_BEGIN# upd_unj 

/*******★*★***★****★**********■*'*************'*******'*'******************/ 

— Copyright * 1997, Epiphany Marketing Software, Inc. All Rights Reserved 
-- upd_unj 

-- Epiphany Marketing Software 

— Update all dimension keys to 'UNKNOWN' in staging table 

— where referential integrity fails 

/******★*****★********************************************************/ 
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y** *********************************************************** ********/ 

— Count the number of rows to update in the staging table - that is, those 

— that have at least one Foreign key where referential integrity fails 
^******** # ************************************************************/ 

— #BLOCK_BEGIN# CountUn] 

BEGIN 

INSERT INTO adaptive_template_prof ile (token_name, number_rows) 

SELECT ' PROCESSED ' , COUNT (1) FROM OrderStage 

INSERT INTO adaptive_template_prof ile (token_name, number_rows) 

SELECT 'MODIFIED', COUNT (1) 

FROM 

OrderStage s 
WHERE 1=0 

OR NOT EXISTS (SELECT 1 FROM ProgramMap_B m_Q4 WHERE m_04.iss = s.iss AND m_04 .program_sskey - 
program_sskey) 

OR NOT EXISTS (SELECT 1 FROM ApplicationMap_B m_03 WHERE m_03.iss = s.iss AND 
m_03 .application_sskey = application_sskey) 

OR NOT EXISTS (SELECT 1 FROM TerritoryMap_B m_06 WHERE m_06.iss = s.iss AND 
m 06. territory_sskey = territory_sskey) 

OR NOT EXISTS (SELECT 1 FROM ProductMap_B m_02 WHERE m_02.iss = s.iss AND m_02 .product_sskey - 

product sskey) t . 

OR NOT EXISTS (SELECT 1 FROM CustomerMap_B m_05 WHERE m_05.iss = s.iss AND m_05 . cus tome r_ss key 

= customershipto_sskey) 

OR NOT EXISTS (SELECT 1 FROM CustomerMap_B m_01 WHERE m_01.iss = s.iss AND m_01 . customer_sskey 
= customerbillto_sskey) 

OR NOT EXISTS (SELECT 1 FROM WarehouseMap_B m_07 WHERE m_07.iss = s.iss AND 
m_07 . warehouse_sskey = warehouse_sskey) 

END 

— #BLOCK_END# CountUn] 

/************************************■*■*******'*****'********************/ 

— Update foreign keys where referential integrity fails 

y************* + ************************ *******************************/ 

— #BLOCK_BEGIN# UpdateUnjprogram_sskey 

UPDATE OrderStage SET prog ram_ss key = 'UNKNOWN' 

WHERE NOT EXISTS (SELECT 1 FROM ProgramMap_B m 

WHERE m.iss = OrderStage . iss AND m.program_sskey = OrderStage .program_sskey) 

— #BLOCK_END# UpdateUn jprogram_sskey 

— #BLOCK_BEGIN# UpdateUnjapplication_sskey 

UPDATE OrderStage SET application_sskey = 'UNKNOWN' 

WHERE NOT EXISTS (SELECT 1 FROM ApplicationMap_B m 

WHERE m.iss = OrderStage . iss AND m. application_sskey = OrderStage . application_sskey) 

— #BLOCK_END# UpdateUnjapplication_sskey 

— #BLOCK_BEGIN# UpdateUnj territory_sskey 

UPDATE OrderStage SET territory_sskey = 'UNKNOWN' 

WHERE NOT EXISTS (SELECT 1 FROM TerritoryMap_B m 

WHERE m.iss = OrderStage . iss AND m. territory_sskey - OrderStage . territory_sskey) 

--#BLOCK_END# UpdateUn j territory_sskey 

— #BLOCK_BEGIN# UpdateUnjproduct_sskey 

UPDATE OrderStage SET product_sskey - 'UNKNOWN' 

WHERE NOT EXISTS (SELECT 1 FROM ProductMap_B m 

WHERE m.iss = OrderStage . iss AND m.product_sskey = OrderStage ,product_sskey) 

— #BLOCK END# UpdateUn jproduct sskey 
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tr;i! 


— #BLOCK_BEGIN# UpdateUn j customershipto_sskey 


UPDATE OrderStage SET customershipto_sskey = ’UNKNOWN' 

WHERE NOT EXISTS (SELECT 1 FROM CustomerMap_B m 

WHERE m.iss = OrderStage . iss AND m.customer_sskey = OrderStage . customershipto_sskey) 
— #BLOCK_END# UpdateUn j customershipto_sskey 


— #BLOCK_BEGIN# UpdateUn;] customerbillto_sskey 
UPDATE OrderStage SET customerbillto_sskey = 'UNKNOWN' 

WHERE NOT EXISTS (SELECT 1 FROM CustomerMap_B m , 

WHERE m.iss = OrderStage . iss AND m.customer_sskey = OrderStage . customerbillto_sskey) 

— #BLOCK_END# UpdateUn j customerbillto_sskey 

— #BLOCK_BEGIN# UpdateUn j warehouse_sskey 

UPDATE OrderStage SET warehouse_sskey = 'UNKNOWN' 

WHERE NOT EXISTS (SELECT 1 FROM WarehouseMap_B in 

WHERE m.iss = OrderStage . iss AND m.warehouse_sskey = OrderStage. warehouse_sskey) 

— #BLOCK_END# UpdateUn^ warehouse_sskey 


— #TEMPLATE_END# upd_un] 


Note, additional semantic types and adaptive templates can be imported into the system 100. 


Method and Apparatus for Creating a Well- PATENT 

Formed Database System Using a Computer Page 1 54 

Attorney Docket No 20308 702 
ODMA\PCDOCS\SQL 1\2280 1 7\1 


Inventors: Craig D. Weissman, 
Greg V. Walsh and Eliot L. Wegbreit 




H \PRIVATBCLIENT\20308 Epiphany\701 MetaSchemas - EpiCenter System\Figure 01 

System vsd 




""N 


> 


Build Datamart 
Process 202 


J 


V. Extraction Process 


r 


204 


} Build 

Aggregates 

205 


I Query and Reporting 
> Process 

[ 206 


J 


Figure 2 

H \PRIVATBCLIENT\20308 Epiphany\701 MetaSchemas - EptCenter System\Figure 02 Overview 

Ftowchartvsd 












: 3ase 
Dimeni ions 
Definiions 
710 


0- {£ Servers 

B — & yea 

3 - 23 Epicenters 
3 - © epitest 
f 0- C-J Base Dimensions 
Account 


< 




r 


Constellations j 
712 ' 


P Application 
; CostCenter 
Customer 
Date 
Entity 
FormatPos 
GlTranstype 
Product 
Program 
Project 
Sub Account 
Territory 
Transtype 

B- 23 Constellations 
’ £r Expense 

& ~C2 Aggregates 
t S Dimensions 

} B - £3 Dimensions 

a- OO Facts 
S-~ £3 Meades 

S - O Aggregates 
B - HJ Dimensions 

Customers IJTo 
) Product 
\ Application 
I Program 
) CustomerShipTo 
) Territory 

~ - fi3 Dimensions (Degenerate 1 
B #3 Facts • 



Sales Constellation 
720 


Sales Aggregates 
721 




■llrl! Order 

SeliThrcMonth 
"Jill SelThruWeek 
S~ O Measures' 


l S~ Q Ticksheets^ 

K -<§> Extraction 
B -29 Security 


Sales Dimensions 
723 

Sales Degenerate 
Dimensions 

725 

Sales Facts 

726 

Sales Measures 

728 

Sales Ticksheets 
729 


Enterprise 
Manager 
Interface 192 


i 


Extraction Definitions 
740 



Figure 7 


K\PRIVATE\CUEMT\20308 Epiphany\701 MetaSchemas * Epicenter System\Figure07 satesconstelaton vsd 



Enterprise 
Manager 
Interface 192 


Base 

Dimension 

810 


Dimension 

820 


Dimension 
Table Definition 
Window 
800 


Figure 8 


R\PRIVATE\CLIEWT\20308 Epphany\701 MetaSchemas - Epicenter System\Figure08 djmrotedefmrttoncustomerbitto vsd 



V Epicenter Enterprise Manager 

--y ^^i^&Jk^*£h^7z£ r *$A£ '£/ -» -vj £ " tus- 



Ei 
M 

Interface 192 


Q Servers 
B fiyeti 

5 - ^£j Epicenters 
E- © eptest 

2- -£j Base Dimensions 

( Account 
Application 
CostCenter 

Customee 

Date 

Entity 

FormatPos 

GITranstype 

Product 

Program 

Project 

SobAccount 

Territory 

Transtype 

3 ^j| Constellations 
Q-- 4& Expense 


Dimension 


Dimension 


Jstowly Changing Dimensions 


Dimension 
Data Semantic 
1 910 


9 base name 


9 regtonjcode 
9 region.name 
9 tier_name 
9 typejcode 
9 type_name 








Dimension 

Columns 

920 


Figure 9 


H:\PRlVATE\CLIEtfn20308Epiphany\701 MetaSchemas* ^(Center System1Figure09customerdimbasedefvsd 



















3 C-J Epicenters 
B @1 epitest 


Enterprise 
Manager 
Interface 192 



Fact Table 
Window 


1300 



Fact Column 

Window 

1400 


Fact Column 
1410 

Physical Type 
1420 

Aggregate 

Operation 

1430 


Figure 14 


H \PFUVATE\CLIENT\20308 Epphany\701 Meta Schemas - EpiCenter System\Figure14 tactcddefiniboavsd 



Enterprise 
I ✓ Manager 
^ Interface 192 


Fact Table 

Window 

1300 

Fact Table 
1310 

Fact Data 
Semantic 
1320 


Fact Columns 
1330 



Figure 15 


H.\PRfVATE\CUENT\20308 Epiphany\701 MetaSchemas - EpiCenter Systefri\figure15 (actehoeeofsemantictype vsd 





B -M. vet' 

E Cl Epicenters 
B~ @epitest 

E Oj Base Dimensions 
) Account 
Application 
) CostCenter 
1J Customer 
I Date 
€3 Entity 


I FormatPos 
\ GLTranstype 
| Product 
l Proyam 
I Project 
t SubAccount 
( Territory 
I Transtype 
^3 Constellations 
1 B" Expense 

H- CD Aggregates 
* 6J - £3 Dimensions 

i S ~ Cl Dimensions (Degenerate) 

Ep - Q] Facts 

i f S- £3 Measures 
3-Jfiricksheets 

S - ‘ “ 


S - OS Aggregates 


Batch Operation: Generating Datamait Schema 

i : ” ! 


|Successf uly generated the Polowing objects: 




Icreated: Account_0_A 
.fCreated: Account _0_B 
treated: AccountStage 
l Created: AccountMap_A 
Created: AccountMapJi 
Created: Application_0_A 
Created: AppIication_0_B 
Created: ApplicationStage 
Created: Applicat»onMap_A 
l^fCreated: Applicat>onMap_B 




Enterprise 
Manager 
Interface 192 


Batch 

Operation 

Window 

1600 


Figure 16 


K\PRIVATE\CUENT>20308 Epiphany\701 Meta Schemas - EprCenter Syster^Figurel 6 generating schema, vsd 



T ". S.u zL^i 




Enterprise 
Manager 
Interface 192 


Job Definition 

Window 

1700 


Job Steps 
1810 



Figure 18 


H \PfWATBCLIENT\20308 Epiphany\701 MetaScbemas - EpiCenter System\Figure18jol>steps vsd 





E- Epicenters 


Enterprise 
Manager 
Interface 192 


Base Dimensions 
| Account 
\ Applcation 
| CostCenter 
| Customer 
\ Date 
) Entity 
FormatPos 
5 GLTranstype 
j Product 
i Pro-am 
\ Project 
I SubAccount 
\ Territory 
I Transtype 
1=3 -^j Constellations 
0™ Expense 
, . &i £3 Aggregates 

* i S— CD Dimensions 



Connector 

Definition 

Window 

1900 


Figure 19 


H \PRIVATE\CUENT\20308 Epphany\701 MetaSchemas - Epicenter SysfemtFJgure19 connectordefinitorTwrthdatastores.vsd 




Enterprise 
Manager 
Interface 192 



Data Store 

Window 

2000 


Figure 20 


H \PRIVAT0CUENT\2O3O8 Epiphany\701 MetaSchemas - EpiCenter System\Rgure20 datasourcedefinrtwrtvsd 



m 



B 9 Servers 
B yeti 

B - Hj Epicenters 
B-- @ epitest 

B "£j Base Dimensions 



Manager 
Interface 192 



All Steps 
Window 
2100 


Figure 21 


K\PRIVATE\CLIENT\20306 Ep«phany\701 MetaSchemas - EpiCenter System\figure21 sqtoonnectorsteps.vsd 



Enterprise 
Manager 
Interface 192 


All Steps 
Window 
2100 

SQL Statement 
Window 
2200 


SQL Field 
2210 


^SELECT 

ca_addr cusfco»er_j*k«y, 

I SJJULL (ca_sort, 'UNKNOWN' J 
ISNULL (ca^typa, 'UNKNOWN') cyp«_cod®, 

I SHULL (type_code_*str . code_amt, UNKNOWN ' ) type__name, 

ISNULL (ca_ragioa, ' UNKNOWN 1 ) regium_cod# , 

ISNULL (rog±on_code_nistr . cod«_cjaat , ’UNKNOWN') ragionjnana, 

'Tiarl* cier_na»e, 

CONVERT ( VARCHAR, CONVERT ( DATBTIHB, c»_*od_d»t® ) ) d*fc«_aodifxed 

IJfroh 

(ca_astr LIFT OUTER JOIN coda_astr typ*_cod«_astr ON 
t-yp®_cod*_»str . codo_f ldnaaa » ' ca_typ« ' AND 
^yp®_coda_iastr. code_vaiue = ca_typa) 

LEFT OUTER JOIN codajastr region_code_msfcr ON 

regian_code_astr . coda_£ldn*ae ■ 'ca_region* AND 
ragxon_code_a*t r . code_vaJ.ua = ca_ragxon 


■ 




Figure 22 


R\PRIVATBCUENT\20308 Epphany\701 MetaSchemas - Epicenter System\Figure22 customersql vsd 














Enterprise 
Manager 
Interface 192 


All Steps 
Window 
2100 

SQL Statement 
Window 
2200 


ISILECT 

*od_ttbr ♦ *-' + sod_liiv« ss_kay, 

CONVERT( CHAR (11) , CONVERT! SKA1LDATETIHB, *°_ord_date) > dat«Jt«y, 

CASI VHEN SUBSTRING ( sod_nbr, 1,1 ) - *R* THEM 2 XLSB 1 END traastyp«_k«y,| 

2 procass_k*y, 

so_cust- customs rbil It o_ssk«y, 
sod_part product_ssk«y, 
so_chaxmsl applic»tlon_**k«y, 

*od_prodlin* prospr*m_s*k«y, 

*o_cust custom# rsbipto_ssk«y, 
so_slspsn81 t«rritory_ssk*y, 

CONVERT ( VARCHAR, CONVERT ( HONEY, 

(CONVERT < RIO AT, sod_qty_ord ) - CONVERT ( RIO AT, sod_qty_fhip ) + CONVERT ( f 
CONVERT ( FLOAT, sodjrica ) / COVERT ( FLOAT, SO_«*_rat« ))) net_prica, 

CASE 

■ him sod__eypa ■ *H‘ THIN 'O' 

ELSE CONVERT (VARCHAR, CONVIRT(DICIHALU6,4> , sod_qty_ord) - 

CONVERT (DECIMAL (16,4 ) , tod_qty_ship) ♦ CONVERT (DECUIAL (16,4) , sad_qty_i»v> ) 
END numb«r_unit.* 

|FROH ” 

sod d«t 



Figure 23 


H \PRIVATE\CLIENT\20308 Epfchaiy170t MetaSchemas - EptCenter System\Rgure23 ordersql.vsd 








Enterprise 
Manager 
Interface 192 


All Steps 
Window 
2100 


All Semantics 

Connector 

2410 



Figure 24 


H \PRIVATBCLIENT\20308 Epiphany1701 MetaSchemas - EpfCenter System\Rgure24 allsemanteccorwiectorandsteps vsd 




s - 9 Servers 
B- A yet' 

2 Epicenters 
S epitest 

E Base D'^erstons 
Account 
\ Application 
\ CostCenter 
\ Customer 
i Date 
\ Entity 
I FormatPos 
i GLTranstype 
\ Product 
\ Program 
| Project 
\ SuhAccount 
| Territory 


Enterprise 
Manager 
Interface 192 


All Steps 
Window 
2100 



All Semantics 

Connector 

2410 


Semantic 

Transformation 

Window 

2500 

Customer 

Semantic 

2510 


Figure 25 

H:\PRIVATE\CUENR20308 Epiphany\701 MetaScbemas - EpiCenter Sy$tem\Figure25 customerseman tic instance vsd 




Enterprise 
Manager 
Interface 192 


s 


-H yeti 

E - Epicenters 


Eh- © epitest 

B~ Base Dimensions 



Account 

Application 

CostCenter 

Customer 

Date 

Entity 

FormatPos 

GLTranstype 

Product 

Program 

Project 

SubAccount 

Territory 


I; 



All Steps 
Window 
2100 



All Semantics 

Connector 

2410 


Semantic 

Transformation 

Window 

2500 

Order Semantic 
2610 


Figure 26 


H:\PRIVATE\CUENT\20308 Ep)phany\701 MetaSchemas - Epicenter $y$tem\Figure26 ordersemantanstance vsd 



Enterprise 
Manager 
Interface 192 







a 



Batch 

Operation 

Window 

1600 


, l±l~ Ticksheets 
E •••••$ Sales 

S~ £3 Aggregates 
Q - £3 Dimensions 

CustomerBilTo 
) Product 
) AppScation 
) Program 
) CustomerShpTo 
) Territory 


— £3 Dimensions (Degenerate) 
B "O Facts 




New Dimension 
2700 


Figure 27 


H \PRIVATBCLIENT\20308 Epiphany\701 Meta Schemas - Epicenter System\Figure27 rrxxfifyschema.vsd 


V Epicenter Enterprise Manager 



Enterprise 
Manager 
Interface 192 


Aggregate 
Group Window 
2800 


Aggregate 
Type List 
2810 


Figure 28 


H \PRIVATBCLIENT\20308 Epiphany\701 MetaSchemas - EprCenter System\Figure28 aggdimtypes vsd 




Enterprise 
Manager 
Interface 192 


Configuration 

Window 

2900 

Transaction 

Types 

2910 


Figure 29 


H.\PRIVATBCUENT\20308 Eprphany1701 MetaScfiemas - EprCenter System\Rgure29 transtypedef.vsd 







Figure 30 


R\PR(VATBCUENT\20308 EptphanytfOl MetaSchemas - Epicenter System\Figure30 measured ritmvsd 











Browser 182 



V/heayou create a report, you wiE get a table with rows and c ehunns and fa cts m eachceB. ’ 
M* : , — n-r— i-T— ' ,i ‘‘ rv ■ ' ■ *'’?; ^ 



il 

m 

Kihejrom 

(downtbt tide) 5 s " 

(> > t & 1 IJ 1 , 1 

■ fffffl 

~“TT -sf— 

In the column*: ' 

(aero (S the top) t ^ V < ~ 

EB 

■ 

Httt - - 

1 ^ ^ 

' f j | Business Unit „ M 4 , ^ < 

* 1 Fiscal Year 

EjSBBBm 
\ pgsyp 

3 > \ ^ 1 , Jf:f ’ x t < f ' i 1 ' 

I For tbe^iact* m each ctS, select at least one from each cohtrmx below V 

* „ P dollar Amoikt: 1 '' R Gros/ s >P BookedfatMM* ' f < 

' ' / ,R TW tV t PHNet 

,< -■ ^ P Stopped by Midi v 

1 V-RASP 1 * '< ^ 

" P SeD-Throtv%b 4? , 



i ^ 

C ReVehus Adpistments 5 > , 1 *, 



' , 

P Totai Rcverme 1 5 






Query/Results 

Interface 

184 


Query Form 
3200 


Figure 32 


H:\PRIVATBCLIEtm20308 Epiphary\701 MetaSchemas - EprCenter System\Figure32 
mulbpiemeasureselectionsusenntef^ce.vsd 




Measure 


Mapping 


Window 

3300 


Figure 33 


H.\PRIVATE\CUEhFn20308 EptphanyffOl MetaSchemas - EptCenter System\Rgure33 measuredefimtion.vsd 










Browser 182 



J- Ihfterow* 

Rl-ZI ^’(downlhtjida) > > 

f " * ; 1 * 

* In the cohanns: )S \f< 
} h ..1.1..L !: ;; (across th* top) 

r'**' 1 ^ a >f 


" J ' [Fiscal Year Jj|> <, 5 

1 For the facte m each celL select at least one froco each cotuma below. ^ 

1 'BStSBaiH „ t i ‘ 1 1 ' f 

> , P ^Collar Amount 5 , t“ looked }■■ v*>P Grossj * - M 

T Uints ,,u 1 F jSbicned < V? Net^C’ " - 

, /P ASP r iBacklofi !.r>r Returns ! J> 

: ic ~ — ^ 


After making choices above, cldc the CREATE REPORT button m the lower left; or 6m select some v 
Filters or Display Options fromtfaemenu m the upper left Tou ciaa $sve yourchoices for later use with ft* 
blue buttons above CREATE REPORT, > '' , f * 

i*> ( ; J ^ ^ f f ' ^ t , 


Query/Results 

Interface 

184 


Query Form 
3200 


Figure 34 


H:\PRIVATBCLIENT\20308 Epphany\701 MeteSchemas - Epicenter System\Figure34 saiesticketsfieetvsd 


Browser 182 



" Quftoww^ * ; 

, l 5'^ . *■ ^F««4lY»*rs , ../!<. . | 

i?w t ms 

^ ms A 

1997 , ||, 1998 

Total ; i 

LIA Customer Name Here, / ^ , i <,$3,762K 

" $25.168K! 

$27,704Kj , S15.954R. 

SXXXXXX^ 

□A Customer Name Here [\i \\' 

! $2 ( 242K; 

S2;235Kj. 

$7,457K[* S15.964K: 

SXXX>0£X •• 

FIa Customer Name Here >* > ' 

^SCKK'* S592K 

$6,5#^ 

t7.457Kfi'. S15.9&4K 

sxxxxxx 

QA Customer Name Here 

1 1 < $3,762K : 

» S5.228K 

$5,60€K| j $15,685K 

sxxxxxx-i 

^IA Customer Name Here ' ' 1 

i, $2,2*2^ 

S5J49K 

S7 ,457k! 815.964R 

sxxx, XXX 

■ ■A Customer, Niuncriere 

I S592K 

* 1 $25,1 68K 

S5,50$k!| $15 JSS 

sxxx^xx 

jriA Customer Name Here 

, ( . i0:, S^ok 

. " $2,235KJ 

S7,224Ki 1 S15964K 

sxxxxxx^ i 

tlA Customcr Name Here ’> 

■i $1,890K 

A " ■; $6,537R 

$2,3?# d; S15.685K. 

sxxx^xxx •* 

IlA Customer Name Here 

1 St, 10 IK 

m , $6.228K| 

$5.85 1K^ ' 1 $3,56 IK 

sxkx^cxx 

riA Customer Name Here 


\ * J 

$3,439k| , S4.233K 

sxxxxxx 

Remaining 10498 < vV/' 

sxxx mxx.xxx 

SXXXiXXX 

sxxx.xxxj ; sxxx^xx. 

sxxx,xxx 

, :io5 Srh' irt- 

*xxx4 sxxxxxx 

sxxxxxx 

SXXX.XXXW$XXX JCXX ; 

sxxx,xxx 

FI Check Top 10 £| Select AE 10508 f ^ << < > ^ 

mmm fe-- mibbi 

Customer 







Query/Results 

Interface 

184 

Results 
3500 


Figure 35 


H \PRIVATBCUENT\20308 Epipbany\701 MetaSchemas - EptCenter System\Rgure35 aajftyTableOutputvsd 


Browser 182 


Llonty fo« Expenses ■ Netscape 



l i «*» epiphany 

I & ■ j 



The Mowing DISFLAY OPTIONS determine hoWtbe fact* are presented 


Currency amount: jC Miltons (M) ^Thousanc^tK) C Ones fit _J, 

Include sums: ;f< of rows (at ngh!) F 7 of columns (across bottom) ' ! > ^ " 

Percentages: 

Tj eachrow.totals 100% O each column totals 100% ' ® nether; ' 

Precision: i ^ 

t? 0 decimal places C T^iecnttal place C 2 decimal places <i? \ 

Include charts: ; > ' 

<? 3D C 2 D C No V’ m 

Which rows: , 

C Top ,5 G Top lO O Top 20 O Top 50 C Top 100 C AH 

Sort rows: < ^ 

G by amount ,C,lby row name' > ! .V ? 

' ' > 1 t N Lj " ^ ^ 


4 

I 

t 

a 


H| You c an limit your reportstoonly a part of the data by nsing the FILTERS below ^ ^ 

J< ^ ! ' ' S '» 1 * S V 1 

(R All Fiscal Yfran ,. ,<?/,,< V Vu< i ~ ■ 

fc -. ! |r 1996 !C 1»7 “ • ,|ni»B ' _ .\3Sk 

», : jSe * C.W* - ft***** **-, rniPmsm^Ar^m^rn^m^-fm 


Query/Results 

Interface 

184 


Options Form 
3600 


Figure 36 


H .IPRfVATHCL I ENT\20 308 Epcphany\701 MetaSchemas * EpiCenter $ystem\Figure36 ticks bee topbons vsd 



IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 


In re Application 


Inventor(s): 

Serial No.: 
Filed: 


Craig David Weissman, Gregory Vincent 
Walsh, Eliot Leonard Wegbreit 

Not Assigned 

Filed Herewith 


Title: Method and Apparatus for Creating a 

Well-Formed Database System Using a 
Computer 


) PATENT APPLICATION 
) 

) 

) 

) 

> 

> 

> 

) 

> 

) 

) 

.) 


DECLARATION FOR PATENT APPLICATION 


As a below named inventor, I hereby declare that my residence, post office address and 
citizenship are as stated below next to my name; I believe that I am the original, first and sole inventor (if 
one name is listed below), first and joint inventor (if plural names are listed below) of the subject matter 
which is claimed and for which a patent is sought on the invention entitled: 

METHOD AND APPARATUS FOR CREATING A WELL-FORMED 
DATABASE SYSTEM USING A COMPUTER 

the specification of which (check applicable ones): 

X is attached hereto; 

was filed with the above-identified "Filed" date and "Serial No." 

was amended on (or amended through) . 

The present application is a utility application of Prior Provisional Application, Application No. 

, filed: and may be considered to disclose and claim subject matter in 

addition to that disclosed in the Prior Application, and I hereby claim the benefit of 35 U.S.C. 
Section 119(e). 

I hereby state that I have reviewed and understand the contents of the above-identified 
specification, including the claims, as amended by any amendment(s) referred to above. I acknowledge 
the duty to disclose information which is material to the examination of the application in accordance 
with Title 37, Code of Federal Regulations, §1.56, including information which became available 
between the filing date of the Prior Application and the national or PCT international filing date of the 
present application. 

I hereby declare that all statements made herein of my own knowledge are true and that all 
statements made on information and belief are believed to be true, and further that these statements were 
made with the knowledge that willful false statements and the like so made are punishable by fine or 
imprisonment, or both, under §1001 of Title 18 of the United States Code and that such willful false 
statements may jeopardize the validity of the application or any patent issuing thereon. 
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(1) Full name of sole 
or first inventor: 


CRAIG DAVID WEISSMAN 


(1) Residence: . 


735 Old Countv Road. Apartment C 


Belmont. CA 94002 


(1) Post Office Address: Same As Above 

(1) Citizenship: United States. 



(1) Inventors signature: 


(1) Date: . 


*************************************************************** 


(2) Full name of second 

joint inventor: GREGORY VINCENT WALSH 

(2) Residence: 16000 Montebello Road 

Cupertino. CA 95014 

(2) Post Office Address: Same As Above 



(3) Full name of third 

joint inventor: HI .TOT LEONARD WEGBREIT 

(3) Residence: 1516 Dana Avenue 

Palo Alto. CA 94303 

(3) Post Office Address: Same As Above 
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Title 37. Code of Federal Regulations. §1.56 


SECTION 1.56. DUTY TO DISCLOSE INFORMATION 
MATERIAL TO PATENTABILITY 


(a) A patent by its very nature is affected with a public 
interest. The public interest is best served, and the most 
effective patent examination occurs when, at the time an 
application is being examined, the Office is aware of and 
evaluates the teachings of all information material to 
patentability. Each individual associated with the filing 
and prosecution of a patent application has a duty of 
candor and good faith in dealing with the Office, which 
includes a duty to disclose to the Office all information 
known to that individual to be material to patentability as 
defined in this section. The duty to disclose information 
exists with respect to each pending claim until the claim is 
canceled or withdrawn from consideration, or the 
application becomes abandoned. Information material to 
the patentability of a claim that is canceled or withdrawn 
from consideration need not be submitted if the 
information is not material to the patentability of any claim 
remaining under consideration in the application. There is 
no duty to submit information which is not material to the 
patentability of any existing claim. The duty to disclose all 
information known to be material to patentability is 
deemed to be satisfied if all information known to be 
material to patentability of any claim issued in a patent was 
cited by the Office or submitted to the Office in the manner 
prescribed by § § 1 .97(b)-(d) and 1.98.* However, no 
patent will be granted on an application in connection with 
which fraud on the Office was practiced or attempted or the 
duty of disclosure was violated through bad faith or 
intentional misconduct. The Office encourages applicants 
to carefully examine: 

(1) prior art cited in search reports of a foreign patent 
office in a counterpart application, and 

(2) the closest information over which individuals 
associated with the filing or prosecution of a patent 
application believe any pending claim patentably 
defines, to make sure that any material information 
contained therein is disclosed to the Office. 

(b) Under this section, information is material to 
patentability when it is not cumulative to information 
already of record or being made of record in the 
application, and 


(1) It establishes, by itself or in combination with 
other information, a prima facie case of 
unpatentability of a claim; or 

(2) It refutes, or is inconsistent with, a position the 
applicant takes in: 

(i) Opposing an argument of unpatentability 
relied on by the Office; or 

(ii) Asserting an argument of patentability. 

A prima facie case of unpatentability is established when 
the information compels a conclusion that a claim is 
unpatentable under the preponderance of evidence, burden- 
of-proof standard, giving each term in the claim its 
broadest reasonable construction consistent with the 
specification, and before any consideration is given to 
evidence which may be submitted in an attempt to establish 
a contrary conclusion of patentability. 

(c) Individuals associated with the filing or prosecution of 
a patent application within the meaning of this section are: 

(1) Each inventor named in the application; 

(2) Each attorney or agent who prepares or prosecutes 
the application; and 

(3) Every other person who is substantively involved 
in the preparation or prosecution of the application 
and who is associated with the inventor, with the 
assignee or with anyone to whom there is an 
obligation to assign the application. 

(d) Individuals other than the attorney, agent or inventor 
may comply with this section by disclosing information to 
the attorney, agent, or inventor. 


* §§ 1 .97(b)-(d) and 1 .98 relate to the timing and manner in 
which information is to be submitted to the Office. 


*** *************************************************** ********* 
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